Center-Based Group and Home-Based Individual Exercise Programs Have Similar Impacts on Gait and Balance in People With Multiple Sclerosis: A Randomized Trial (2021), PragMS

1. Eligibility criteria: Who is selected to participate in the trial?

Quotes:

• “Community dwelling people with MS were recruited through the Multiple Sclerosis Society of New South Wales and South Australia.”

• “Inclusion criteria were the following: diagnosis of MS made by a neurologist, able to walk 10 meters with or without an aid within 2 minutes, no relapse of their MS in the past 4 weeks, and no other neurological or orthopedic condition that would affect their function.”

Reasoning: The eligibility criteria for this trial are clearly defined and specific. Participants must have a diagnosis of MS, be able to walk a certain distance within a specified time frame, and meet other criteria related to relapse and other conditions. The criteria are relevant to the population of interest and are likely to be similar to those used in usual care. [Score: 4]

2. Recruitment: How are participants recruited into the trial?

Quotes:

• “Community dwelling people with MS were recruited through the Multiple Sclerosis Society of New South Wales and South Australia.”

• “Participants were assessed at two points at baseline (8 weeks apart) to establish stability of presentation, at the end of an 8-week training period and at 8-week follow-up.”

Reasoning: Participants were recruited through the Multiple Sclerosis Society, which suggests that the trial aimed to recruit participants who are representative of the population of interest. The use of multiple assessment points at baseline also indicates that participants were likely recruited over a period of time rather than all at once, which increases the generalizability of the trial results. [Score: 4]

3. Setting: Where is the trial being done?

Quotes:

• “The exercise program for both groups included two, 60-minute sessions per week, held at least 2 days apart for 8 weeks.”

• “The group program was conducted at a local community center and included four to six participants per class.”

Reasoning: The trial was conducted in both a community center and participants’ homes, which suggests that the trial aimed to replicate the usual care setting for people with MS. The use of a community center for the group program also indicates that the trial was designed to be applicable to a range of settings. [Score: 4]

4. Organisation: What expertise and resources are needed to deliver the intervention?

Quotes:

• “The center-based groups of four to six were supervised by a physiotherapist.”

• “Home exercise participants were provided with one center-based physiotherapist-led session at the commencement of the intervention period for training, and telephone support by a physiotherapist to discuss their program every 2 weeks.”

Reasoning: The intervention was delivered by physiotherapists, which suggests that the trial aimed to use existing healthcare staff and resources that would be available in usual care. The use of telephone support for the home-based group also indicates that the trial aimed to provide support and resources that are feasible in usual care. [Score: 4]

5. Flexibility (delivery): How should the intervention be delivered?

Quotes:

• “Exercises were aimed at improving gait speed, endurance, and balance. They were individualized, specific, progressed in load, and performed at a moderate to high intensity.”

• “Interventions focused on body weight, functional strengthening, part or full functional task execution, balance exercises with reduced base of support, altered surfaces, and internal perturbations.”

Reasoning: The intervention was described as individualized and specific, which suggests that the trial aimed to replicate the usual care approach to delivering interventions. The focus on functional tasks and the use of various exercises and techniques also indicate that the trial aimed to provide flexibility in how the intervention was delivered, similar to usual care. [Score: 4]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

Quotes:

• “Participants in both groups were encouraged to maintain their usual activity levels and engagement in any other exercise throughout the trial, with any variation self-reported to the treating physiotherapist.”

• “Home exercise participants were provided with one center-based physiotherapist-led session at the commencement of the intervention period for training, and telephone support by a physiotherapist to discuss their program every 2 weeks.”

Reasoning: The trial did not impose strict measures to ensure adherence to the intervention, but instead encouraged participants to maintain their usual activity levels and report any variations to the treating physiotherapist. The provision of training and telephone support for the home-based group also indicates that the trial aimed to provide resources and support that are feasible in usual care. [Score: 4]

7. Follow-up: How closely are participants followed up?

Quotes:

• “Participants attended a local community center to complete all assessments at pre-intervention (week 0), post-intervention (week 8), and at follow-up (week 16).”

• “Two baseline assessments were conducted 8 weeks apart to evaluate the stability of clinical presentation, with presentation considered stable if there was no significant difference between baseline assessments.”

Reasoning: Participants were followed up at multiple time points, including pre-intervention, post-intervention, and follow-up. The use of multiple assessments and the evaluation of stability of clinical presentation indicate that the trial aimed to closely follow participants over time, similar to usual care. [Score: 4]

8. Primary outcome: How relevant is it to participants?

Quotes:

• “The primary outcome measure was gait speed, measured using the 10-meter walk test.”

• “Secondary outcome measures included walking endurance, measured using the 6-minute walk test, and standing balance, using the Berg Balance Scale.”

Reasoning: The primary outcome measure of gait speed is directly relevant to participants with MS, as it is a key measure of mobility and function. The use of secondary outcome measures related to walking endurance and balance further supports the relevance of the outcomes to participants. The measures used in the trial are likely to be similar to those used in usual care. [Score: 5]

9. Primary analysis: To what extent are all data included?

Quotes:

• “Data were analyzed according to the intention-to-treat principle.”

• “Differences between groups at baseline were analyzed using analysis of variance (ANOVA) for continuous data and Mann-Whitney U tests for interval data.”

Reasoning: The trial used an intention-to-treat analysis, which includes all available data and is consistent with a pragmatic approach. The use of appropriate statistical tests to analyze differences between groups at baseline further supports the inclusion of all data in the analysis. [Score: 5]

Group-Based Individualized Comprehensive Core Stability Intervention Improves Balance in Persons With Multiple Sclerosis: A Randomized Controlled Trial. (2019), PragMS

1. Eligibility criteria: Who is selected to participate in the trial?

• “Eighty people with Expanded Disability Status scores of 1 to 6.5 participated in this trial.”

The full text provides information on the eligibility criteria for the trial, stating that participants had Expanded Disability Status scores of 1 to 6.5. This suggests that the trial included individuals with a range of disability levels, which is in line with a pragmatic approach.

Score: [4]

2. Recruitment: How are participants recruited into the trial?

• “In August 2015, 1 of the MS nurses at the Department of Neurology, Nordland Hospital Trust in Bodø, Norway, sent out invitation letters with a consent form to 160 persons with MS who were registered at the MS outpatient clinic, had EDSS scores of 0 to 7, and lived in 1 of the 6 municipalities included in the study.”

The full text describes the recruitment process, which involved sending invitation letters to individuals with MS who were registered at the MS outpatient clinic and lived in the specified municipalities. This approach is likely to increase the applicability of the trial results.

Score: [4]

3. Setting: Where is the trial being done?

• “The GroupCoreDIST intervention was conducted by 6 physical therapists in 6 municipalities in Norway. Standard care included the usual care for individuals with MS in the same municipalities.”

The full text provides information on the setting of the trial, stating that it was conducted in 6 municipalities in Norway. This suggests that the trial was conducted in a real-world setting, which is in line with a pragmatic approach.

Score: [4]

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “Six neurological physical therapists conducted the intervention after being trained in GroupCoreDIST for 5 days.”

The full text mentions that the intervention was conducted by neurological physical therapists who received training in GroupCoreDIST. This suggests that the intervention was delivered by healthcare professionals with expertise in the specific intervention, which is in line with a pragmatic approach.

Score: [4]

5. Flexibility (delivery): How should the intervention be delivered?

• “The physical therapists individualized the exercises by tailoring them according to each individual’s symptoms, disability level, and general well-being, and they intensified the exercises by increasing number of repetitions, level of difficulty, and/or by adding motor-cognitive dual tasks as movement quality improved.”

The full text describes how the intervention was delivered, stating that the physical therapists individualized the exercises based on each participant’s symptoms, disability level, and general well-being. This suggests that the intervention was flexible and tailored to the individual, which is in line with a pragmatic approach.

Score: [4]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• “Participants in the intervention group were encouraged not to seek any physical therapy other than GroupCoreDIST during the 6 weeks of the intervention.”

The full text mentions that participants in the intervention group were encouraged not to seek any physical therapy other than GroupCoreDIST during the 6 weeks of the intervention. This suggests that there were no strict measures in place to enforce adherence to the intervention, which is in line with a pragmatic approach.

Score: [5]

7. Follow-up: How closely are participants followed up?

• “Assessments were conducted at baseline and at weeks 7, 18, and 30.”

The full text mentions that assessments were conducted at baseline and at weeks 7, 18, and 30. This suggests that participants were followed up at multiple time points, which is in line with a pragmatic approach.

Score: [5]

8. Primary outcome: How relevant is it to participants?

• “The objective of this study was to compare the immediate and long-term effects of a 6-week individualized, group-based, comprehensive core stability intervention (GroupCoreDIST) with standard care on balance and trunk control in individuals with MS.”

The full text clearly states that the objective of the study was to compare the effects of the intervention on balance and trunk control. This suggests that the primary outcome is directly relevant to the participants, which is in line with a pragmatic approach.

Score: [5]

9. Primary analysis: To what extent are all data included?

• “All participants with postintervention test assessment scores were included, although some had missing observations.”

The full text mentions that all participants with postintervention test assessment scores were included in the analysis, even if they had missing observations. This suggests that the primary analysis included all available data, which is in line with a pragmatic approach.

Score: [5]

A double-blinded randomized controlled trial of laparoendoscopic single-site access versus conventional 3-port appendectomy (2012), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

• “Consecutive patients admitted for suspected acute appendicitis, aged between 18 and 75 years, were recruited.”

• “The exclusion criteria were as follows: diagnosis of appendicitis was not clinically established (symptoms attributable to urinary or gynecological problems), symptoms of more than 5 days, and/or a palpable mass in the right lower quadrant suggesting an appendiceal mass.”

• “Additional patients with the following conditions were also excluded: history of cirrhosis and coagulation disorders, generalized peritonitis, shock upon admission, previous abdominal surgery, ascites, suspected or proven malignancy, contraindication to general anesthesia (severe cardiac and/or pulmonary disease), inability to give informed consent, and pregnancy.”

The eligibility criteria for this trial appear to be quite pragmatic. They include consecutive patients admitted for suspected acute appendicitis, which is likely to reflect the population that would be eligible for the intervention in usual care. The exclusion criteria also seem reasonable and are likely to be similar to what would be considered in usual care. Score: [4]

2. Recruitment: How are participants recruited into the trial?

• “The patients were randomized after surgery was offered. They were randomly allocated to LESS appendectomy (group A) or TPLA (group B) by opening sealed envelopes containing a computer-generated code in blocks of 10.”

The recruitment method used in this trial is not explicitly described in the full text. It is mentioned that patients were randomized after surgery was offered, but it is unclear how patients were identified and approached for participation. Without more information, it is difficult to assess the recruitment domain. Score: [NA]

3. Setting: Where is the trial being done?

• “The study was performed between October 2009 and March 2011 in the Prince of Wales Hospital (University affiliated) and the North District Hospital.”

The trial was conducted in two hospitals in Hong Kong. The description of the setting is limited, but it appears that the trial was conducted in hospital settings that provide government-supported services to the local population. Without more information, it is difficult to assess the match between the trial setting and the setting where the results are likely to be applied. Score: [NA]

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “All procedures were performed or supervised by specialist surgeons with experience of more than 20 LESS appendectomies. They were also proficient with advanced laparoscopic surgical techniques and had experience of more than 100 laparoscopic procedures in their respective subspecialties.”

The expertise and resources needed to deliver the intervention are described in the full text. The surgeons performing the procedures had experience with LESS appendectomies and advanced laparoscopic surgical techniques. This suggests that the intervention could be delivered by surgeons with expertise in laparoscopic surgery, which is likely to be similar to the expertise available in usual care. Score: [4]

5. Flexibility (delivery): How should the intervention be delivered?

• “Two 5-mm ports and one 10-mm port were inserted through a 13-mm transumbilical incision via multiple fascial incisions.”

• “Conversion would be performed for the following reasons: unclear anatomy, compromised safety of dissection, complications including bleeding or adjacent organ injury, or severe contamination.”

The delivery of the intervention is described in the full text. The surgical technique involves the insertion of ports through a transumbilical incision. The criteria for conversion to a different surgical technique are also specified. The description suggests that there is some flexibility in how the intervention is delivered, as the decision to convert is based on clinical judgment. Score: [3]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• “Postoperative analgesics were standardized. All patients were given regular combination tablets (acetaminophen 500 mg and phenyltoloxamine citrate 30 mg–-Dologesics, Llorens Pharmaceuticals International Division, Miami, FL) and supplementary intravenous tramadol hydrochloride (0.7 mg/kg every 4 hours, equivalent to 10 times morphine dosage) if the oral medications were inadequate for pain control.”

The measures to ensure participant adherence to the intervention are not explicitly described in the full text. It is mentioned that postoperative analgesics were standardized, but there is no information on how adherence to the medication regimen was monitored or enforced. Without more information, it is difficult to assess the adherence domain. Score: [NA]

7. Follow-up: How closely are participants followed up?

• “They were then required to follow-up at 2 weeks for reassessment.”

The follow-up period for this trial is limited to 2 weeks. There is no information on the frequency or intensity of follow-up visits or assessments during this period. The short follow-up duration suggests that participants may not be closely followed beyond the immediate postoperative period. Score: [3]

8. Primary outcome: How relevant is it to participants?

• “The primary outcome was the overall pain score. This was defined as the most severe pain experienced by the patient within their last 24 hours and was measured by the visual analogue scale on a 10-cm scale.”

The primary outcome of overall pain score is directly relevant to the participants, as it measures their pain experience after the surgery. The use of a visual analogue scale is a common and validated method for assessing pain. Score: [5]

9. Primary analysis: To what extent are all data included?

• “All outcomes were analyzed according to the intention-to-treat principle.”

The primary analysis of this trial was conducted according to the intention-to-treat principle, which means that all randomized participants were included in the analysis regardless of their adherence to the assigned intervention. This approach is consistent with a pragmatic trial design. Score: [5]

Impact of Thoracoscopic T2 Sympathectomy on Patients With Primary Palmar and Axillary Hyperhidrosis (2018), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

Quotes:

• “From October 2012 to August 2016, a total number of 120 patients (of primary palmar and axillary hyperhidrosis) were enrolled in this prospective randomized study.”

• “Patients were randomized preoperatively and divided into 2 groups: group A (underwent T2, T3, and T4 sympathectomy) and group B (underwent T3 and T4 sympathectomy) with 1 year of follow-up.”

Reasoning: The full text provides information on the eligibility criteria for the trial, stating that patients with primary palmar and axillary hyperhidrosis were included. The patients were randomized into two groups based on the level of sympathectomy they underwent. The text does not provide information on any specific exclusion criteria or reasons for excluding certain patients. Overall, the eligibility criteria appear to be pragmatic, including patients with the condition of interest and randomizing them to different levels of intervention. Score: [4]

2. Recruitment: How are participants recruited into the trial?

Quotes:

• “Between October 2012 and August 2016, a prospective study was designed with a total number of 120 patients with primary palmar and axillary hyperhidrosis after approval from the Institutional Ethics Committee.”

• “Patients were randomly divided into 2 groups: group A and group B. Randomization was done by using a computer program called the Random Number Generator.”

Reasoning: The full text mentions that the study was prospective and that patients were recruited over a specific time period. It also states that patients were randomly divided into two groups using a computer program. However, there is no information provided on how patients were identified or approached for recruitment. The text does not mention any specific recruitment strategies or efforts made to increase participation. Based on the available information, it is difficult to assess the recruitment approach. Score: [NA]

3. Setting: Where is the trial being done?

Quotes:

• “Between October 2012 and August 2016, a prospective study was designed with a total number of 120 patients with primary palmar and axillary hyperhidrosis after approval from the Institutional Ethics Committee.”

Reasoning: The full text does not provide specific information on the setting where the trial was conducted. It only mentions that the study was designed and approved by an Institutional Ethics Committee. Without further details, it is not possible to assess the match between the trial setting and the setting where the results are likely to be applied. Score: [NA]

4. Organisation: What expertise and resources are needed to deliver the intervention?

Quotes:

• “Bilateral and sequential sympathectomy were performed in all cases in the same setting.”

• “Double lumen endotracheal intubation with general anesthesia is the standard technique.”

• “Sympathectomy was carried out by electrocoagulation hook at levels T2, T3, and T4 in group A and at level T3 and T4 in group B.”

Reasoning: The full text provides information on the surgical technique used to deliver the intervention, including the use of bilateral and sequential sympathectomy, double lumen endotracheal intubation, and electrocoagulation hook. However, there is no information provided on the expertise or resources needed to deliver the intervention beyond the standard surgical techniques. It is unclear if any additional training or resources were required. Based on the available information, it is difficult to assess the match between the organization of care in the trial and usual care. Score: [NA]

5. Flexibility (delivery): How should the intervention be delivered?

Quotes:

• “Sympathectomy was carried out by electrocoagulation hook at levels T2, T3, and T4 in group A and at level T3 and T4 in group B.”

Reasoning: The full text provides information on the specific technique used to deliver the intervention, which involves the use of an electrocoagulation hook at specific levels of the sympathetic chain. There is no mention of any flexibility in the delivery of the intervention or any variation in the technique used. Based on the available information, it appears that the delivery of the intervention was highly specified and protocol-driven. Score: [2]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

Quotes:

• “Patients were classifed into 4 groups: very satisfied, satisfied, partially satisfied, and not recommending sympathectomy to other patients.”

• “All patients were asked to put themselves in 1 of these 4 groups in follow-up visits or by direct telephone contact after 6 and 12 months postoperatively using a visual analog scale (VAS).”

Reasoning: The full text mentions that patients were classified into different groups based on their satisfaction with the intervention. However, there is no information provided on any specific measures in place to ensure adherence to the intervention. It is unclear if any strategies were used to monitor or enforce adherence. Based on the available information, it is difficult to assess the flexibility in adherence to the intervention. Score: [NA]

7. Follow-up: How closely are participants followed up?

Quotes:

• “Data on patient reported outcomes were collected by regular visits in outpatient clinic and by direct telephone contact.”

• “There were no missing data for these variables.”

Reasoning: The full text mentions that data on patient reported outcomes were collected through regular visits in the outpatient clinic and direct telephone contact. It also states that there were no missing data for the variables collected. However, there is no information provided on the frequency or duration of follow-up visits or the completeness of follow-up. Based on the available information, it is difficult to assess the extent of follow-up. Score: [NA]

8. Primary outcome: How relevant is it to participants?

Quotes:

• “All patients had improvement of hand sweating.”

• “Overdry hands were significant in group A.”

• “Compensatory sweating was significant in group A after the first month, after 6 months, and after 12 months.”

Reasoning: The full text provides information on the primary outcome of the trial, which is the improvement of hand sweating. It also mentions specific outcomes related to the primary outcome, such as overdry hands and compensatory sweating. These outcomes are directly relevant to the participants and their experience of hyperhidrosis. Based on the available information, it appears that the primary outcome is of obvious importance to the participants. Score: [5]

9. Primary analysis: To what extent are all data included?

Quotes:

• “Results are given as mean ± SD.”

• “Significance between the 2 groups was determined by chi-square test comparing patients’ data, complications, and patients’ satisfaction.”

Reasoning: The full text provides information on the statistical analysis of the data, including the use of mean and standard deviation for continuous variables and chi-square test for comparing patients’ data. However, there is no information provided on the inclusion of all available data in the analysis or any specific handling of missing data. Based on the available information, it is difficult to assess the extent to which all data were included in the analysis. Score: [NA]

The Home-based Older People’s Exercise (HOPE) trial: a pilot randomised controlled trial of a home-based exercise intervention for older people with frailty (2014), PragQoL

1. Eligibility criteria: Who is selected to participate in the trial?

Quotes:

• “Eligibility criteria included living at home and receiving case manager care, being housebound or attending day centres in Bradford, UK.”

• “People were excluded if they were unable to stand and walk independently; currently participating in an alternative exercise programme; registered blind. Those who had poorly controlled angina; had another household member already in the trial; had severe dementia or were receiving palliative care, were also excluded.”

Reasoning: The eligibility criteria for this trial seem to be quite pragmatic. They include a range of individuals who are likely to be candidates for the intervention in usual care, such as those living at home, receiving case manager care, and attending day centers. The exclusion criteria also seem reasonable and are likely to be based on safety concerns or potential interference with the intervention. Overall, the trial appears to have a pragmatic approach to eligibility criteria.

Score: [4]

2. Recruitment: How are participants recruited into the trial?

Quotes:

• “Verbal assent was first sought by the member of the health or social services team coordinating individual care. Potential participants identified from GP registers were contacted by letter from the practice and responses were mailed directly to the study team.”

• “Those expressing an interest were visited at home by trained clinical researchers who undertook detailed screening, explained the study in full to eligible candidates and provided written information leaflets. Informed consent was obtained prior to baseline assessment.”

Reasoning: The recruitment methods used in this trial appear to be pragmatic. The trial recruited participants from various sources, including GP registers, and used a combination of verbal assent and written consent. The process of visiting potential participants at home and providing detailed information about the study suggests an effort to recruit participants in a way that is similar to usual care. Overall, the recruitment methods seem to align with a pragmatic approach.

Score: [4]

3. Setting: Where is the trial being done?

Quotes:

• “The trial was conducted in Bradford, UK.”

• “People were recruited from various settings, including those living at home, receiving case manager care, and attending day centers.”

Reasoning: The trial was conducted in a specific location (Bradford, UK), which may limit the generalizability of the results to other settings. However, the trial recruited participants from various settings within Bradford, including those living at home and attending day centers, which suggests an effort to include a diverse population that is representative of the usual care setting. While the setting may not be ideal for generalizability, the trial seems to have made pragmatic design choices within the chosen setting.

Score: [3]

4. Organisation: What expertise and resources are needed to deliver the intervention?

Quotes:

• “The HOPE programme is a 12-week-progressive exercise intervention that is presented to participants in an exercise manual and delivered by community-based physiotherapists.”

• “Physiotherapists receive intervention training in a 2-h workshop. At the beginning of the intervention, participants are requested to perform five repetitions of each exercise in the routine. This progresses to 10 and then 15 repetitions as performance improves.”

Reasoning: The delivery of the intervention in this trial seems to align with a pragmatic approach. The intervention is delivered by community-based physiotherapists, which suggests that it can be integrated into the usual organization of care. The training provided to the physiotherapists is relatively brief, and the progression of the exercise routine is based on individual performance, which indicates flexibility in the delivery of the intervention. Overall, the trial seems to have a pragmatic approach to the organization of care.

Score: [4]

5. Flexibility (delivery): How should the intervention be delivered?

Quotes:

• “The HOPE programme is graded into three levels. Participants are stratified to the appropriate level using their baseline performance on the timed-up-and-go test (TUGT).”

• “The exercises require no special equipment and can be performed without professional supervision.”

Reasoning: The delivery of the intervention in this trial appears to be quite flexible. The intervention is graded into different levels based on individual performance, which allows for tailoring to the needs of each participant. The exercises can be performed without professional supervision and do not require special equipment, which suggests that they can be easily implemented in usual care. The trial seems to have a pragmatic approach to the delivery of the intervention.

Score: [4]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

Quotes:

• “Adherence diaries were returned by 27 of the 28 participants (96%) who completed the 12-week intervention.”

• “Total adherence was defined as the percentage of days that the exercises were completed three times a day for 5 days over the course of the 12-week intervention.”

Reasoning: The trial does not explicitly mention any measures to enforce adherence to the intervention. The use of self-completed adherence diaries suggests that participants were encouraged to track their adherence, but there is no mention of any consequences for non-adherence. The trial seems to have a more pragmatic approach to adherence, allowing for flexibility in how participants engage with the intervention.

Score: [4]

7. Follow-up: How closely are participants followed up?

Quotes:

• “Follow-up information is available for 70 participants (83% of those randomised; 40 intervention participants and 30 control participants).”

• “All outcomes were collected at baseline and at 14 weeks post-randomisation.”

Reasoning: The follow-up rate of 83% suggests that participants were reasonably well followed up in this trial. The follow-up period of 14 weeks is relatively short but seems appropriate for assessing the outcomes of interest. The trial does not mention any additional contacts or visits beyond the baseline and follow-up assessments. Overall, the trial seems to have a pragmatic approach to follow-up.

Score: [4]

8. Primary outcome: How relevant is it to participants?

Quotes:

• “The primary outcome was mobility, measured using the timed-up-and-go test (TUGT).”

• “The TUGT was developed as a basic mobility test for older people and has good accuracy for identifying frailty.”

Reasoning: The choice of the primary outcome, mobility measured using the TUGT, seems to be relevant to the participants in this trial. The TUGT is a validated measure of mobility and is likely to be important to older people with frailty. The trial seems to have a pragmatic approach to the choice of the primary outcome.

Score: [4]

9. Primary analysis: To what extent are all data included?

Quotes:

• “Missing data were addressed by listwise exclusion.”

• “The final intention-to-treat analysis included all randomised participants for whom the follow-up assessment of the primary outcome measure was available.”

Reasoning: The trial used an intention-to-treat analysis, which includes all randomised participants for whom the follow-up assessment of the primary outcome measure was available. However, the trial also mentions listwise exclusion for missing data, which suggests that participants with missing data were not included in the analysis. This approach seems to be more explanatory than pragmatic, as it may introduce bias and limit the generalizability of the results.

Score: [3]

Hybrid Minimally Invasive Esophagectomy for Esophageal Cancer (2019), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

Quotes:

• “We performed a multicenter, open-label, randomized, controlled trial involving patients 18 to 75 years of age with resectable cancer of the middle or lower third of the esophagus.”

• “Specific inclusion criteria were the following: squamous-cell carcinoma or adenocarcinoma of thoracic esophagus with a clinical stage of I, II, or III (tumor stage 1 through 3 [T1, T2, or T3], no nodal involvement [N0] or presence of cancer in lymph nodes [N1] or in distant lymph nodes [≥5 cm from the tumor; N2], and no metastases [M0]) before the receipt of any induction treatment; esophageal cancer in the middle or lower third of the esophagus or junctional (Siewert’s type I) tumor; the receipt or nonreceipt of neoadjuvant radiotherapy, chemotherapy, or both; tumors that were considered to be resectable with a curative intention at the time of preoperative evaluation; an age of 18 to 75 years; a World Health Organization performance-status score of 0, 1, or 2 (on a 5-point scale, with higher numbers indicating greater disability); ability to provide written informed consent; ability to undergo one of the investigated surgical procedures; and ability to attend the follow-up visits.”

Reasoning: The eligibility criteria for this trial are clearly defined and include specific characteristics such as age, tumor stage, and performance status. The criteria are relevant to the population of interest and are likely to be similar to those used in usual care. The trial does not exclude any specific groups that would be likely to receive the intervention in usual care. Score: [5]

2. Recruitment: How are participants recruited into the trial?

Quotes:

• “The trial was conducted at 13 centers in France.”

• “The trial protocol has been published previously13 and is available at NEJM.org. The trial was funded by the French National Cancer Institute.”

• “Randomization was performed centrally, with the use of the stratified-field block-randomization method (blocks of four) for each participating center.”

Reasoning: The trial was conducted at multiple centers in France, which suggests that participants were recruited from different clinics and increases the generalizability of the trial results. The trial protocol was published and the trial was funded by a national cancer institute, indicating transparency and support for the study. Randomization was performed centrally, ensuring a fair allocation of participants. Score: [4]

3. Setting: Where is the trial being done?

Quotes:

• “The trial was conducted at 13 centers in France.”

• “The trial protocol has been published previously13 and is available at NEJM.org. The trial was funded by the French National Cancer Institute.”

Reasoning: The trial was conducted in multiple centers in France, which suggests that the trial setting is representative of the usual care setting for patients with esophageal cancer in France. The trial protocol was published and the trial was funded by a national cancer institute, indicating transparency and support for the study. Score: [4]

4. Organisation: What expertise and resources are needed to deliver the intervention?

Quotes:

• “All the participating centers had surgeons who were experienced in esophageal-cancer surgery and in laparoscopic gastric mobilization, with at least 25 procedures performed at each center before it was included in the trial.”

• “A technical surgical video of the abdominal minimally invasive phase was sent to each participating center to standardize surgical technique, and during the initial cases, surgical technical supervision was undertaken by the principal investigator at each center, as stipulated in the protocol.”

Reasoning: The participating centers had surgeons with experience in esophageal-cancer surgery and laparoscopic gastric mobilization, which suggests that the intervention was delivered by healthcare professionals with the necessary expertise. The trial also implemented measures to standardize surgical technique and provide technical supervision, ensuring consistency in the delivery of the intervention. Score: [4]

5. Flexibility (delivery): How should the intervention be delivered?

Quotes:

• “After laparoscopic abdominal exploration and confirmation of tumor resectability, patients were randomly assigned to undergo hybrid minimally invasive esophagectomy (hybrid-procedure group) or open esophagectomy (open-procedure group). Despite the difference in approach to the abdominal component of the surgical procedure, the surgical technique was standardized between groups and performed as described previously.”

• “No pyloric drainage procedures were routinely performed in either group, and the anastomosis was either sutured or stapled at the discretion of the operating surgeon.”

Reasoning: The trial protocol specified the surgical techniques for both the hybrid minimally invasive esophagectomy and open esophagectomy groups, ensuring consistency in the delivery of the intervention. However, there was some flexibility in the specific details of the surgical technique, such as the method of anastomosis, which was left to the discretion of the operating surgeon. Score: [3]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

Quotes:

• “All the patients were scheduled to receive a transthoracic en bloc esophagectomy with an extended two-field lymphadenectomy and terminolateral anastomosis in the upper chest with the use of a gastric conduit in all cases.”

• “Guidelines for enhanced recovery after surgery were followed at all the participating centers.”

Reasoning: The trial protocol specified the surgical procedure that all patients were scheduled to receive, ensuring adherence to the intervention. Additionally, guidelines for enhanced recovery after surgery were followed, which suggests that measures were in place to optimize patient adherence to postoperative care. Score: [4]

7. Follow-up: How closely are participants followed up?

Quotes:

• “Patients were followed up at 30 days after surgery and every 6 months for 3 years after surgery.”

• “Follow-up included physical examination; thoracoabdominal CT scan every 6 months; an ear, nose, and throat examination once per year; bronchoscopy for squamous-cell cancer every 2 years; and esophagogastroduodenoscopy every 2 years.”

Reasoning: Participants were followed up at regular intervals for a period of 3 years after surgery, which is likely to capture important outcomes and provide a comprehensive assessment of the intervention’s effects. The follow-up included various examinations and imaging tests, ensuring thorough monitoring of participants. Score: [4]

8. Primary outcome: How relevant is it to participants?

Quotes:

• “The primary end point was major complication during surgery or within 30 days after surgery. A major intraoperative and postoperative complication was defined as a surgical or medical complication with a Clavien–Dindo grade of II or higher (this five-grade system includes subgrades in grades III and IV, and higher grades indicate more life-threatening complications).”

• “Secondary end points included postoperative death within 30 days, intraoperative and postoperative overall complications (major and minor) within 30 days, major pulmonary complications within 30 days, disease-free survival, and overall survival.”

Reasoning: The primary outcome of major complications during or within 30 days after surgery is directly relevant to participants as it captures the occurrence of significant surgical or medical complications. The secondary outcomes, including postoperative death, overall complications, major pulmonary complications, disease-free survival, and overall survival, are also important outcomes that are likely to be meaningful to participants. Score: [5]

9. Primary analysis: To what extent are all data included?

Quotes:

• “All the analyses were performed on an intention-to-treat basis; the analyses included all the patients who had undergone randomization, regardless of the surgery performed and eligibility criteria.”

• “Because the initial statistical analysis plan did not include a provision for correcting for multiple comparisons when tests for secondary or other end points were conducted, P values were not provided, and results are reported as point estimates with 95% confidence intervals.”

Reasoning: The primary analysis was performed on an intention-to-treat basis, which includes all patients who underwent randomization, regardless of the surgery performed or eligibility criteria. This approach ensures that all available data are included in the analysis and reflects the real-world scenario of patients receiving the intervention. Score: [5]

Randomized clinical trial of posterior retroperitoneoscopic adrenalectomy versus lateral transperitoneal laparoscopic adrenalectomy with a 5-year follow-up (2014), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

• “Patients referred to the Third Department of General Surgery, Jagiellonian University Medical College, Krakow, between January 2006 and June 2008 for first time adrenal surgery were registered. Eligible patients with adrenal tumors were assessed for the study.”

• “The inclusion criterion was planned unilateral adrenal surgery for a benign tumor up to 7 cm in diameter.”

• “Exclusion criteria were diffuse peritonitis in history, major abdominal surgery in history, planned bilateral adrenal surgery, adrenal tumor more than 7 cm in diameter, suspected adrenocortical cancer, metastasis to adrenal gland, previous adrenal surgery, pregnancy or lactation, age less than 18 or more than 80 years, American Society of Anesthesiologists fitness grade IV, and inability to comply with the follow-up protocol.”

• “The patients were randomized to one of the parallel groups: PRA or LTLA.” [Score: 4] The full text provides information on the eligibility criteria for the trial, including the inclusion and exclusion criteria. It also mentions that patients were randomized to one of the parallel groups, indicating that the trial aimed to include a diverse population similar to usual care. However, it does not provide specific details on the adherence or follow-up requirements for the participants, which could impact the score.

2. Recruitment: How are participants recruited into the trial?

• “Patients referred to the Third Department of General Surgery, Jagiellonian University Medical College, Krakow, between January 2006 and June 2008 for first time adrenal surgery were registered.”

• “The patients were randomized to one of the parallel groups: PRA or LTLA.” [Score: 4] The full text mentions that patients were referred to the hospital for adrenal surgery and were registered for the study. It also states that the patients were randomized to one of the parallel groups, indicating that recruitment was likely done through usual care channels. However, it does not provide information on any additional recruitment strategies or efforts made to ensure diverse representation, which could impact the score.

3. Setting: Where is the trial being done?

• “Patients referred to the Third Department of General Surgery, Jagiellonian University Medical College, Krakow, between January 2006 and June 2008 for first time adrenal surgery were registered.” [Score: 5] The full text clearly states that the trial was conducted at the Third Department of General Surgery, Jagiellonian University Medical College in Krakow. This indicates that the trial was conducted in a specific setting and provides information on the location, which is important for assessing the applicability of the trial results.

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “All operations in this study were unilateral total adrenalectomies and were carried out by a single experienced endocrine surgeon (M.B.) under general anesthesia.”

• “The surgeon (M.B.) completed a training program in LTLA during his visiting observership in 2002 in the Mount Sinai School of Medicine, Minimally Invasive Surgery Center (New York) headed by M. Gagner, and in PRA during his visiting observership in 2003 in the Department of Surgery and Center for Minimally Invasive Surgery, University of Essen (Essen, Germany) headed by M. Walz.”

• “The surgeon (M.B.) has been performing adrenalectomies by either PRA or LTLA since 2003, and before the starting date of this randomized study, he gained proficiency with both approaches on the basis of 107 consecutive adrenalectomies (50 PRA and 57 LTLA).” [Score: 4] The full text provides information on the expertise and resources needed to deliver the intervention, including the qualifications and training of the surgeon who performed the operations. It also mentions that the surgeon had experience with both PRA and LTLA prior to the study. However, it does not provide specific details on the resources or additional training required beyond the usual care setting, which could impact the score.

5. Flexibility (delivery): How should the intervention be delivered?

• “All operations in this study were unilateral total adrenalectomies and were carried out by a single experienced endocrine surgeon (M.B.) under general anesthesia.”

• “The same anesthesia protocol was used in all the patients.”

• “The detailed surgical technique of both PRA and LTLA has been described in the supplementary material available online.” [Score: 4] The full text provides information on the delivery of the intervention, including the type of surgery performed and the use of a standardized anesthesia protocol. It also mentions that the detailed surgical technique for both PRA and LTLA is available in supplementary material. However, it does not provide specific details on the flexibility or variability in the delivery of the intervention, which could impact the score.

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• “The patients, nurses, and clinical investigators assessing outcomes in this study were blinded to the relevant group assignment.”

• “The patients were encouraged to be mobilized and to drink and eat soon after returning from the recovery room.”

• “As soon as the patient tolerated 750 mL of clear liquids, the diet was advanced to a solid diet.” [Score: 4] The full text mentions that the patients, nurses, and clinical investigators were blinded to the group assignment, indicating that adherence to the intervention was likely monitored. It also mentions that the patients were encouraged to mobilize and consume liquids and solid food after surgery. However, it does not provide specific details on any additional measures or interventions in place to ensure adherence to the intervention, which could impact the score.

7. Follow-up: How closely are participants followed up?

• “The patients were encouraged to be mobilized and to drink and eat soon after returning from the recovery room.”

• “The prevalence of postoperative complications was evaluated during hospitalization and visits at the outpatient surgical department scheduled at 7 days and 1, 12, 24, 36, 48, and 60 months after the operation.” [Score: 4] The full text mentions that the patients were encouraged to mobilize and consume liquids and solid food after surgery, indicating that follow-up was likely conducted to assess postoperative recovery. It also mentions that postoperative complications were evaluated during hospitalization and at scheduled outpatient visits up to 60 months after the operation. However, it does not provide specific details on the frequency or intensity of follow-up, which could impact the score.

8. Primary outcome: How relevant is it to participants?

• “The primary endpoint was duration of surgery.”

• “The secondary endpoints were intraoperative blood loss, conversion rate, postoperative pain, prevalence of shoulder-tip pain, additional analgesia requests, nausea and vomiting, time to oral intake, time to ambulation, length of hospital stay, total cost of the operation, postoperative complications (including long-term surgical access site herniation and need for hernia repair), and in cases of surgery for hormonally active tumors also biochemical and clinical cure rate during 5-year follow-up.” [Score: 4] The full text mentions that the primary endpoint of the trial was the duration of surgery, which is directly relevant to the participants. It also provides information on several secondary endpoints that are important for assessing the effectiveness and impact of the intervention. However, it does not provide specific details on the selection or justification of the primary outcome measure, which could impact the score.

9. Primary analysis: To what extent are all data included?

• “Data are presented as mean with 95% confidence interval (CI) unless stated otherwise.”

• “The analysis of primary and secondary variables was performed on an intention-to-treat basis.” [Score: 4] The full text mentions that the data were analyzed on an intention-to-treat basis, indicating that all available data were included in the analysis. It also mentions that the results are presented with confidence intervals, providing a measure of uncertainty. However, it does not provide specific details on any additional analyses or handling of missing data, which could impact the score.

Thoracoscopic sympathicotomy for disabling palmar hyperhidrosis: a prospective randomized comparison between two levels (2011), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

• “This study compared sympathicotomy over the second (R2) vs third (R3) costal head relative to these variables in patients with massive palmar hyperhidrosis.”

• “This prospective, randomized study enrolled 121 patients with disabling palmoplantar hyperhidrosis assigned to bilateral sympathicotomy (sympathetic transection), which was done over R2 in 61 (n = 122 extremities) or R3 in 60 (n = 120 extremities).”

• “The patients frequently were treated with medical management before they sought consultation for surgical consideration.”

The full text mentions that the study enrolled patients with disabling palmoplantar hyperhidrosis and compared the effectiveness of sympathicotomy over the R2 and R3 costal heads. It also mentions that the patients had received medical management before considering surgery. Based on this information, it can be inferred that the eligibility criteria for the trial included patients with a specific condition (palmoplantar hyperhidrosis) and a history of failed medical management. The full text does not provide information on other eligibility criteria such as age, comorbidities, or adherence to treatment.

Score: [3]

2. Recruitment: How are participants recruited into the trial?

• “This prospective, randomized study enrolled 121 patients with disabling palmoplantar hyperhidrosis assigned to bilateral sympathicotomy.”

• “For those patients without prior medical treatments and whose insurance carriers so required, we initiated topical aluminum chloride hexahydrate 20% (Drysol, Person & Covey, Glendale, CA) or 6.25% (Xerac AC, Person & Covey) in anhydrous ethyl alcohol, oral glycopyrrolate (Robinul, Shionogi USA Inc, Florham Park, NJ) at progressively increased dosages, or iontophoresis with the Drionic device (General Medical, Los Angeles, CA), despite the realization that these medical treatments would likely fail.”

The full text mentions that the study enrolled 121 patients with palmoplantar hyperhidrosis. It also mentions that some patients had received prior medical treatments before considering surgery. However, there is no information provided on the specific recruitment strategies used to enroll participants into the trial, such as the recruitment sites or methods used to identify potential participants.

Score: [NA]

3. Setting: Where is the trial being done?

• “This prospective, randomized study enrolled 121 patients with disabling palmoplantar hyperhidrosis assigned to bilateral sympathicotomy.”

• “All operations were done as outpatient procedures, and no complications developed, including Horner syndrome, bleeding, pneumothorax, or infection.”

The full text mentions that the study was a prospective, randomized study that enrolled patients with palmoplantar hyperhidrosis. It also mentions that the operations were done as outpatient procedures. However, there is no information provided on the specific settings where the trial was conducted, such as the healthcare facilities or geographic locations.

Score: [NA]

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “Simple electrocautery was used to perform a single sympathicotomy transection per side at the R2 or R3 level.”

• “All operations were done as outpatient procedures, and no complications developed, including Horner syndrome, bleeding, pneumothorax, or infection.”

The full text mentions that the intervention (sympathicotomy) was performed using simple electrocautery and that the operations were done as outpatient procedures without any complications. However, there is no information provided on the specific expertise or resources needed to deliver the intervention, such as the qualifications of the healthcare professionals performing the procedure or the availability of specialized equipment or facilities.

Score: [NA]

5. Flexibility (delivery): How should the intervention be delivered?

• “Simple electrocautery was used to perform a single sympathicotomy transection per side at the R2 or R3 level.”

• “Patients were questioned at 6 months and after 1 year by phone or mail survey.”

The full text mentions that the intervention (sympathicotomy) was delivered using simple electrocautery. It also mentions that the patients were followed up at 6 months and after 1 year using phone or mail surveys. However, there is no information provided on the specific details of how the intervention was delivered or any flexibility in the delivery of the intervention.

Score: [NA]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• “Patients were questioned at 6 months and after 1 year by phone or mail survey.”

• “The patients frequently were treated with medical management before they sought consultation for surgical consideration.”

The full text mentions that the patients were followed up at 6 months and after 1 year using phone or mail surveys. It also mentions that some patients had received medical management before considering surgery. However, there is no information provided on any specific measures in place to ensure participants’ adherence to the intervention or any strategies used to monitor or improve adherence.

Score: [NA]

7. Follow-up: How closely are participants followed up?

• “Patients were questioned at 6 months and after 1 year by phone or mail survey.”

• “The patients frequently were treated with medical management before they sought consultation for surgical consideration.”

The full text mentions that the patients were followed up at 6 months and after 1 year using phone or mail surveys. However, there is no information provided on the specific details of the follow-up process, such as the completeness of follow-up or any additional measures taken to ensure participant engagement or response.

Score: [NA]

8. Primary outcome: How relevant is it to participants?

• “This study compared sympathicotomy over the second (R2) vs third (R3) costal head relative to these variables in patients with massive palmar hyperhidrosis.”

• “Patients were questioned at 6 months and at 1 year or more to assess efficacy, side effects, and satisfaction with the procedure.”

The full text mentions that the study compared sympathicotomy over the R2 and R3 costal heads in patients with palmoplantar hyperhidrosis. It also mentions that the primary outcomes assessed were efficacy, side effects, and patient satisfaction with the procedure. These outcomes are directly relevant to the participants as they measure the effectiveness of the intervention and its impact on their symptoms and satisfaction.

Score: [5]

9. Primary analysis: To what extent are all data included?

• “Patients were questioned at 6 months and after 1 year by phone or mail survey.”

• “The Wilcoxon rank sum test was used for differences in continuous variables between sympathicotomy groups.”

The full text mentions that the patients were followed up at 6 months and after 1 year using phone or mail surveys. It also mentions the use of statistical tests to analyze differences in continuous variables between the sympathicotomy groups. However, there is no information provided on the specific methods used for data analysis or any details on the inclusion or handling of missing data.

Score: [NA]

Anterior cruciate ligament reconstruction using patellar tendon versus hamstring tendon: a prospective comparative study with 9-year follow-up (2011), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

Quotes: “Seventy-one patients with an acute rupture of the ACL were prospectively randomized to either BPT or HT for ACL reconstruction.” “Exclusion criteria were any concomitant ligament or meniscus injury or previous surgery, evidence of chondral lesion higher than grade 2, and any damage to the contralateral knee.”

Reasoning: The full text provides information on the eligibility criteria for the trial, including the specific inclusion and exclusion criteria used to select participants. The criteria are clearly defined and help to determine the similarity between the trial population and the population that would receive the intervention in usual care. Score: [4]

2. Recruitment: How are participants recruited into the trial?

Quotes: “Randomization was done by tossing a coin before the first examination.” “Exclusion criteria were any concomitant ligament or meniscus injury or previous surgery, evidence of chondral lesion higher than grade 2, and any damage to the contralateral knee.”

Reasoning: The full text does not provide specific information on how participants were recruited into the trial. It only mentions that randomization was done by tossing a coin. Without further details, it is difficult to assess the recruitment strategy and determine its similarity to usual care. Score: [NA]

3. Setting: Where is the trial being done?

Quotes: “All patients were operated on between October 1998 and September 1999 by the senior author.”

Reasoning: The full text provides information on the time period during which the trial was conducted, but it does not provide specific details about the setting where the trial was done. Without this information, it is difficult to assess the similarity between the trial setting and the setting where the results will be applied. Score: [NA]

4. Organisation: What expertise and resources are needed to deliver the intervention?

Quotes: “All patients were operated on between October 1998 and September 1999 by the senior author.” “An arthroscopic-assisted technique was used for both grafts.”

Reasoning: The full text mentions that the surgical procedures were performed by the senior author, but it does not provide specific information on the expertise and resources needed to deliver the intervention. Without this information, it is difficult to assess the match between how care is organized and delivered in the trial and how the intervention would be made available in usual care. Score: [NA]

5. Flexibility (delivery): How should the intervention be delivered?

Quotes: “Both grafts underwent femoral fixation without any implants by a special press-fit technique and tibial fixation over a bone bridge.” “We hypothesized that anatomic ACL reconstruction with BPT or quadrupled HT graft fixation close to the insertion sites with a femoral tunnel drilled through the anteromedial portal (AMP) results in similar good clinical, functional, and MRI results but less discomfort in the HT group.”

Reasoning: The full text provides information on how the intervention was delivered, including the use of a special press-fit technique for femoral fixation and tibial fixation over a bone bridge. It also mentions the hypothesis that anatomic ACL reconstruction with both grafts would result in similar clinical, functional, and MRI results. This suggests that the delivery of the intervention was flexible and aimed to mimic usual care. Score: [4]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

Quotes: “All patients were operated on between October 1998 and September 1999 by the senior author.” “No significant difference was found in the grade of medial or lateral meniscal lesion or the number of patients having meniscal lesions when the operated and nonoperated knees were compared.”

Reasoning: The full text does not provide specific information on the measures in place to ensure participants adhere to the intervention. It only mentions that all patients were operated on by the senior author. Additionally, it mentions that there were no significant differences in the grade of meniscal lesions between the operated and nonoperated knees. Without further details, it is difficult to assess the flexibility of adherence measures in the trial. Score: [NA]

7. Follow-up: How closely are participants followed up?

Quotes: “At a mean follow-up of 8.8 years (SD, 0.55 years; range, 7.41 to 10 years), 54 patients were examined.” “Unfortunately, we lost 8 patients to follow-up (dropout rate, 12.9%).”

Reasoning: The full text provides information on the follow-up period and the number of patients who were examined. It also mentions that 8 patients were lost to follow-up. This suggests that participants were followed up closely, but the dropout rate of 12.9% is relatively high. Score: [3]

8. Primary outcome: How relevant is it to participants?

Quotes: “The goal of ACL reconstruction is to provide appropriate stability to allow for good function and to protect the knee from developing cartilage damage, meniscus tears, and osteoarthritis.” “The IKDC form showed a significant difference between both groups: 84% patients in the BPT group and 94.4% in the HT group had an IKDC score of A or B, indicating a normal or nearly normal result.”

Reasoning: The full text mentions the goal of ACL reconstruction, which is to provide stability and protect the knee from further damage. It also provides information on the IKDC scores, which indicate the clinical relevance of the primary outcome. This suggests that the primary outcome is relevant to the participants. Score: [5]

9. Primary analysis: To what extent are all data included?

Quotes: “All patients were assessed by 2 independent examiners 9 years postoperatively.” “All MRI scans were analyzed by an independent and blinded examiner, a certified radiologist, who is well experienced and unaware of the clinical and functional results.”

Reasoning: The full text mentions that all patients were assessed by independent examiners and that the MRI scans were analyzed by an independent and blinded examiner. This suggests that all data were included in the analysis. Score: [5]

Simultaneous bilateral stereotactic procedure for deep brain stimulation implants: a significant step for reducing operation time (2016), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

Quotes:

• “57 patients with movement disorders underwent bilateral DBS implantation in the same study period.”

• “After giving informed consent, 57 patients with movement disorders—mostly PD but also dystonia and tremor—underwent bilateral DBS implantation in the same operation.”

Reasoning: The full text mentions that 57 patients with movement disorders were included in the trial, specifically those with Parkinson’s disease, dystonia, and tremor. The eligibility criteria are not explicitly described in the text, but it can be inferred that patients with these movement disorders were eligible for the trial. However, there is no information provided about any specific inclusion or exclusion criteria.

Score: [NA]

2. Recruitment: How are participants recruited into the trial?

Quotes:

• “After giving informed consent, 57 patients with movement disorders—mostly PD but also dystonia and tremor—underwent bilateral DBS implantation in the same operation.”

Reasoning: The full text does not provide any information about how participants were recruited into the trial. It only mentions that patients with movement disorders who gave informed consent underwent the bilateral DBS implantation. There is no mention of any recruitment strategies or efforts.

Score: [NA]

3. Setting: Where is the trial being done?

Quotes:

• “The procedures were performed within the period from June 2012 to December 2014.”

• “Our local ethics committee approved this study.”

Reasoning: The full text mentions that the procedures were performed within a specific time period and that the study was approved by a local ethics committee. However, there is no information provided about the specific setting where the trial was conducted, such as the location, healthcare system, or type of clinic.

Score: [NA]

4. Organisation: What expertise and resources are needed to deliver the intervention?

Quotes:

• “Permanent quadripolar DBS electrodes (model 3387, Medtronic Inc.) were implanted bilaterally in the GPi, the STN, or the Vim using stereotactic planning software–based image guidance.”

• “The operative technique was performed after administering a local anesthetic and consisted of routine steps.”

Reasoning: The full text mentions that the DBS electrodes were implanted using stereotactic planning software and that the operative technique consisted of routine steps. However, there is no information provided about any additional expertise or resources needed to deliver the intervention. It can be inferred that the intervention was delivered using standard techniques and resources.

Score: [4]

5. Flexibility (delivery): How should the intervention be delivered?

Quotes:

• “Permanent quadripolar DBS electrodes (model 3387, Medtronic Inc.) were implanted bilaterally in the GPi, the STN, or the Vim using stereotactic planning software–based image guidance.”

• “The operative technique was performed after administering a local anesthetic and consisted of routine steps.”

Reasoning: The full text does not provide any information about the flexibility in the delivery of the intervention. It only mentions that the DBS electrodes were implanted using stereotactic planning software and that the operative technique consisted of routine steps. There is no mention of any specific details or variations in the delivery of the intervention.

Score: [NA]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

Quotes:

• “The simultaneous bilateral implant procedures were 38.5% faster than the conventional bilateral approach.”

• “Overall, shorter procedures are more comfortable for the patient, who is therefore more likely to cooperate with the clinical examination.”

Reasoning: The full text does not provide any information about specific measures in place to ensure participants adhere to the intervention. However, it mentions that the simultaneous bilateral approach resulted in shorter procedures, which may contribute to increased patient comfort and cooperation. This suggests that there may be a pragmatic approach to adherence in the trial.

Score: [3]

7. Follow-up: How closely are participants followed up?

Quotes:

• “Only 20 patients who had completed 1-year follow-up were considered for this analysis.”

Reasoning: The full text mentions that only 20 patients who had completed a 1-year follow-up were considered for the analysis. However, there is no information provided about the frequency or intensity of the follow-up, or any specific measures taken to ensure participant follow-up. The limited information provided suggests that the follow-up may not be extensive.

Score: [3]

8. Primary outcome: How relevant is it to participants?

Quotes:

• “The patients had a mean improvement of approximately 47% in the motor score UPDRS-III, comparing the off-medication condition with the off-medication/on-stimulation condition 1 year after the procedure.”

Reasoning: The full text mentions that the primary outcome measure was the improvement in the motor score UPDRS-III in patients with Parkinson’s disease. This outcome is relevant to the participants as it assesses the impact of the intervention on their motor symptoms. However, there is no information provided about the selection or justification of this outcome measure.

Score: [4]

9. Primary analysis: To what extent are all data included?

Quotes:

• “To compare motor outcomes between the sequential and simultaneous groups we employed the Mann-Whitney unpaired t-test with significance set at p < 0.05.”

Reasoning: The full text mentions that a Mann-Whitney unpaired t-test was used to compare motor outcomes between the sequential and simultaneous groups. However, there is no information provided about any specific handling of missing data or any other analysis methods used. It can be inferred that all available data were included in the analysis.

Score: [4]

Prospective randomized clinical trial of single- versus double-row suture anchor repair in 2- to 4-cm rotator cuff tears: clinical and magnetic resonance imaging results (2011), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

Quotes:

• “Inclusion criteria for this study were (1) a full-thickness rotator cuff tear proven by preoperative magnetic resonance imaging (MRI) or magnetic resonance arthrography, (2) symptoms lasting more than 6 months with proper conservative treatment, and (3) a 2- to 4-cm tear size in either the coronal or sagittal plane, which was measured and confirmed intraoperatively.”

• “Exclusion criteria were (1) a rotator cuff tear in which either SR or DR repair was not possible; (2) a cuff tear in which complete footprint coverage was impossible; (3) any history of prior surgery, fracture, dislocation, or infection of the affected shoulder; (4) inflammatory joint disease of the affected shoulder, including rheumatoid arthritis; and (5) an open repair, including an open repair of the subscapularis.”

Reasoning: The study clearly defines the eligibility criteria for participants, including the specific criteria for rotator cuff tear size and the exclusion criteria. The criteria are relevant to the condition being studied and are likely to be similar to those used in usual care. The study does not exclude any specific groups of patients that would be likely to receive the intervention in usual care. Score: [5]

2. Recruitment: How are participants recruited into the trial?

Quotes:

• “From December 2005 to March 2007, arthroscopic rotator cuff repair was performed in 184 shoulders. Eighty-eight patients met the inclusion criteria, but seventeen were excluded. A total of 71 rotator cuff tears were repaired with either the SR or DR repair technique. Repair methods were allocated by a simple randomization method after confirmation of inclusion criteria intraoperatively.”

Reasoning: The study describes the recruitment process, which involved performing arthroscopic rotator cuff repair in eligible patients. The repair methods were allocated using a simple randomization method, ensuring that participants were recruited in an unbiased manner. The recruitment process appears to be similar to how patients would be recruited in usual care. Score: [5]

3. Setting: Where is the trial being done?

Quotes:

• “Surgeries were performed by a single surgeon (J.C.Y.) with the patient under general anesthesia in the lateral decubitus position.”

• “The study was conducted at a single institution.”

Reasoning: The trial was conducted at a single institution, which may limit the generalizability of the results to other settings. However, the setting of the trial (a single surgeon performing the surgeries) is likely to be similar to how the intervention would be delivered in usual care. Score: [4]

4. Organisation: What expertise and resources are needed to deliver the intervention?

Quotes:

• “Surgeries were performed by a single surgeon (J.C.Y.) with the patient under general anesthesia in the lateral decubitus position.”

• “Rotator cuff tears were repaired with simple stitches for the SR repairs and the lateral row for the DR repairs. Mattress sutures were used for the medial row of the DR repairs.”

Reasoning: The intervention (rotator cuff repair) was delivered by a single surgeon, which is likely to be similar to how the intervention would be delivered in usual care. The resources and expertise needed to deliver the intervention (surgical skills and anesthesia) are likely to be available in usual care settings. Score: [5]

5. Flexibility (delivery): How should the intervention be delivered?

Quotes:

• “Rotator cuff tears were repaired with simple stitches for the SR repairs and the lateral row for the DR repairs. Mattress sutures were used for the medial row of the DR repairs.”

Reasoning: The study does not provide detailed information on the flexibility of delivering the intervention. However, based on the description of the surgical techniques used, it appears that the intervention was delivered in a specific and standardized manner. This suggests a less flexible approach to delivery, which may be more explanatory than pragmatic. Score: [3]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

Quotes:

• “Rehabilitation protocols were identical for all study subjects.”

• “Routine postoperative outpatient follow-up visits were conducted at 2 weeks, 6 weeks, and 3, 6, 12, and 24 months postoperatively.”

Reasoning: The study does not provide specific information on measures to ensure participant adherence to the intervention. However, the use of identical rehabilitation protocols for all study subjects suggests a standardized approach to adherence. The routine follow-up visits also indicate a level of monitoring and support for adherence. Score: [4]

7. Follow-up: How closely are participants followed up?

Quotes:

• “Postoperatively, MRI was performed at a mean of 27.5 ± 11.2 months after index surgery.”

• “Postoperative follow-up visits were conducted at 2 weeks, 6 weeks, and 3, 6, 12, and 24 months postoperatively.”

Reasoning: The study provides information on the timing of follow-up visits and the use of postoperative MRI at a specific time point. The follow-up period of up to 24 months allows for a reasonable assessment of outcomes. The follow-up procedures appear to be similar to what would be done in usual care. Score: [4]

8. Primary outcome: How relevant is it to participants?

Quotes:

• “The purpose of this study was to compare SR repair with 2 lateral suture anchors and DR repair with an additional medial suture anchor for 2- to 4-cm tears.”

• “The principal finding of this study was that there is no difference between SR and DR rotator cuff repairs in terms of clinical outcome at a mean of 31.9 months after surgery.”

Reasoning: The primary outcome of the study (comparison of clinical outcomes between SR and DR repairs for 2- to 4-cm tears) is directly relevant to the participants and addresses the research question. The study provides evidence that there is no difference in clinical outcomes between the two repair techniques. Score: [5]

9. Primary analysis: To what extent are all data included?

Quotes:

• “Comparisons of demographic data, tear size on preoperative magnetic resonance imaging, global fatty degeneration index, and concomitant procedures showed no differences between the SR and DR groups.”

• “No intergroup differences in pain visual analog scale, American Shoulder and Elbow Surgeons, Constant, or University of California, Los Angeles scores were found at final follow-up.”

Reasoning: The study includes all available data in the analysis, comparing various demographic and clinical variables between the SR and DR groups. The analysis includes outcome measures relevant to the research question. Score: [5]

Cost-effectiveness of Randomized Study of Laparoscopic Versus Open Bilateral Inguinal Hernia Repair (2018), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

• “Cases of primary, reducible bilateral inguinal hernia were included and randomized using a simple randomization program.” [1]

• “Exclusion criteria were contraindications for general anesthesia or laparoscopy, obstructed or strangulated inguinal hernias, and hernia recurrences.” [1]

• “Patients (aged over 18 yrs), with primary, bilateral inguinal hernias assessed by ultrasound were included.” [2]

The full text provides information on the eligibility criteria for the trial. Patients with primary, reducible bilateral inguinal hernias were included, while those with contraindications for general anesthesia or laparoscopy, obstructed or strangulated hernias, and hernia recurrences were excluded. The criteria appear to be pragmatic, as they include patients who would typically be candidates for the intervention in usual care. Score: [4]

2. Recruitment: How are participants recruited into the trial?

• “Cases were randomized using a simple randomization with a computer program and divided into 2 groups according to the surgical approach elected by the computer program.” [2]

The full text mentions that cases were randomized using a simple randomization program. However, it does not provide information on how participants were recruited into the trial. Without this information, it is difficult to assess the recruitment domain. Score: [NA]

3. Setting: Where is the trial being done?

• “This study was carried out in a private hospital in Madrid (Spain) at the General Surgery Department of Sanchinarro University Hospital recruiting patients from March 2013.” [2]

The full text states that the trial was conducted at a private hospital in Madrid, Spain. It does not provide information on the match between the trial setting and the setting where the results are likely to be applied. Score: [3]

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “The same team of surgeons (4 senior consultant surgeons with a minimum of 2 years’ experience in both TAPP and OL techniques) and an anesthesiology team performed the operation.” [3]

The full text mentions that the same team of surgeons with experience in both TAPP and OL techniques performed the operations. It does not provide information on any additional expertise or resources needed to deliver the intervention. Score: [4]

5. Flexibility (delivery): How should the intervention be delivered?

• “All procedures were performed using the same surgical steps as follows.” [3]

• “The same polypropylene mesh used in the TAPP procedure is placed and fixed using prolene sutures.” [3]

The full text indicates that the surgical steps and materials used for the intervention were standardized. It does not mention any flexibility in the delivery of the intervention. Score: [2]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• “A standard analgesia regime was used equally for all patients postoperatively up to 7 days.” [3]

The full text mentions that a standard analgesia regime was used for all patients postoperatively. It does not provide information on any specific measures to ensure adherence to the intervention. Score: [3]

7. Follow-up: How closely are participants followed up?

• “During follow-up, there was a drop out of 6 patients (2 TAPP, 4 OL) and they have been excluded only from the cost-effectiveness analysis.” [3]

The full text mentions that there were dropouts during follow-up, but it does not provide information on the frequency or duration of follow-up. Score: [3]

8. Primary outcome: How relevant is it to participants?

• “QALYs were estimated for 1 year following the procedure for each patient using the medical outcomes study SF-36 questionnaire.” [3]

The full text mentions that quality-adjusted life years (QALYs) were used to measure effectiveness, and the SF-36 questionnaire was used to estimate QALYs for 1 year following the procedure. The primary outcome of QALYs is relevant to participants as it captures their quality of life. Score: [4]

9. Primary analysis: To what extent are all data included?

• “Data has been recorded in a SPSS Statistics Version 20.0 database.” [3]

The full text mentions that data has been recorded in a database, but it does not provide information on the inclusion of all data in the analysis. Score: [3]

Total Hip Arthroplasty or Hemiarthroplasty for Hip Fracture (2019), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

• “We randomly assigned 1495 patients who were 50 years of age or older and had a displaced femoral neck fracture to undergo either total hip arthroplasty or hemiarthroplasty.”

• “All enrolled patients had been able to ambulate without the assistance of another person before the fracture occurred.”

The full text provides clear information about the eligibility criteria for the trial. Patients aged 50 years or older with a displaced femoral neck fracture who were able to ambulate independently before the fracture were included. This pragmatic approach ensures that the trial population closely resembles the patients who would typically receive these interventions in usual care. Score: [5]

2. Recruitment: How are participants recruited into the trial?

• “We enrolled patients at 80 participating sites in Canada, the United States, Spain, the United Kingdom, the Netherlands, Norway, Finland, Australia, New Zealand, and South Africa.”

• “Minimization was used within each center to ensure prognostic balance between the treatment groups with regard to age, prefracture living setting, prefracture functional status, and American Society for Anesthesiologists (ASA) physical status.”

The full text mentions that patients were enrolled at multiple sites in various countries, indicating a diverse recruitment strategy. The use of minimization within each center also suggests an effort to balance the treatment groups based on important prognostic factors. These recruitment approaches align with a pragmatic trial design. Score: [5]

3. Setting: Where is the trial being done?

• “The trial was conducted in 80 centers in 10 countries.”

The full text provides information about the number of centers and countries involved in the trial, but it does not explicitly discuss the match between the trial setting and the setting where the results will be applied. Without further details, it is difficult to assess the impact of the setting on applicability. Score: [NA]

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “Nearly all participating surgeons (97.8%) met thresholds for surgical expertise.”

• “Patients were assigned to undergo either total hip arthroplasty or hemiarthroplasty.”

The full text mentions that participating surgeons met thresholds for surgical expertise, indicating that the trial was conducted by experienced professionals. The intervention involved two surgical procedures, total hip arthroplasty and hemiarthroplasty, which are commonly performed in clinical practice. These aspects suggest a pragmatic approach to the organization of care. Score: [5]

5. Flexibility (delivery): How should the intervention be delivered?

• “Patients who underwent total hip arthroplasty had superior function as measured by the WOMAC total score, pain score, stiffness score, and function score, modestly favored total hip arthroplasty over hemiarthroplasty.”

The full text does not provide explicit information about the flexibility in delivering the intervention. However, the mention of superior function in patients who underwent total hip arthroplasty suggests that the intervention was delivered in a way that allowed for individual variation and adaptation based on patient needs and preferences. This aligns with a more pragmatic approach to delivery flexibility. Score: [4]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• “A total of 54 patients (7.5%) who had originally been assigned to total hip arthroplasty received hemiarthroplasty, and 21 (2.9%) who had originally been assigned to hemiarthroplasty received total hip arthroplasty.”

The full text mentions that a small percentage of patients crossed over from their originally assigned treatment group. This suggests that there was some flexibility in adherence to the assigned intervention, allowing patients to receive the alternative treatment if deemed necessary. However, the text does not provide further details about measures to ensure adherence. Score: [3]

7. Follow-up: How closely are participants followed up?

• “Over 24 months, the functional assessment tests and quality-of-life questionnaires were completed by 943 to 1141 patients (depending on the test) at one or more follow-up visits.”

The full text indicates that functional assessments and quality-of-life questionnaires were completed by a substantial number of patients at multiple follow-up visits over a 24-month period. This suggests a relatively intensive follow-up approach, which may be more explanatory than pragmatic. Score: [3]

8. Primary outcome: How relevant is it to participants?

• “The primary end point was any unplanned secondary hip procedure within 24 months after the initial surgery.”

The primary outcome of the trial, any unplanned secondary hip procedure within 24 months, is directly relevant to the participants and reflects an important clinical outcome. This aligns with a pragmatic approach to selecting a primary outcome that is meaningful to patients. Score: [5]

9. Primary analysis: To what extent are all data included?

• “Analyses included patients in the groups to which they had been randomly assigned. Data for a given patient were censored at 24 months of follow-up or at the time of the last follow-up for patients who were lost to follow-up.”

The full text indicates that analyses included all patients in their assigned treatment groups, regardless of adherence or follow-up status. This intention-to-treat analysis approach ensures that all available data are included and reflects a pragmatic approach to the primary analysis. Score: [5]

Roux-en-Y or Billroth II Reconstruction After Radical Distal Gastrectomy for Gastric Cancer: A Multicenter Randomized Controlled Trial (2018), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

• “We included patients between ages of 21 and 80 years with histological diagnosis of gastric adenocarcinoma who underwent radical DG with curative intent.”

• “Patients who had previous gastrectomy, stomach, or small bowel surgery precluding construction of either form of anastomosis, radiological evidence of carcinomatosis, or who were admitted for emergency gastrectomy for complications related to the tumor, such as perforation, bleeding, and obstruction, were excluded.”

• “The characteristics of the 2 groups are similar.”

The full text provides information on the eligibility criteria for the trial. Patients between the ages of 21 and 80 years with histological diagnosis of gastric adenocarcinoma who underwent radical DG with curative intent were included. Patients with previous gastrectomy, stomach or small bowel surgery that would prevent the construction of anastomosis, radiological evidence of carcinomatosis, or those admitted for emergency gastrectomy for complications related to the tumor were excluded. The text also mentions that the characteristics of the two groups were similar. Score: [4]

2. Recruitment: How are participants recruited into the trial?

• “Between October 2008 and October 2014, a total of 164 patients were recruited for the trial.”

• “Once the patient met the inclusion/exclusion criteria, the patient was registered into the trial and informed consent was obtained.”

The full text mentions that a total of 164 patients were recruited for the trial between October 2008 and October 2014. It also states that patients who met the inclusion/exclusion criteria were registered into the trial and informed consent was obtained. However, there is no information provided on the specific recruitment strategies used or whether any extra resources were required for recruitment. Score: [3]

3. Setting: Where is the trial being done?

• “This is a multicenter prospective RCT conducted at specialized Upper Gastrointestinal (UGI) Surgery units at 4 tertiary hospitals in Singapore and Hong Kong.”

The full text states that the trial was conducted at specialized Upper Gastrointestinal (UGI) Surgery units at 4 tertiary hospitals in Singapore and Hong Kong. This suggests that the trial was conducted in a specialized setting, which may limit the generalizability of the results to other settings. Score: [3]

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “All operations were performed by UGI surgeons who are specialized in gastric cancer surgery.”

• “The techniques of reconstruction were standardized throughout the participating institutions.”

The full text mentions that all operations were performed by Upper Gastrointestinal (UGI) surgeons who are specialized in gastric cancer surgery. It also states that the techniques of reconstruction were standardized throughout the participating institutions. This suggests that the expertise and resources needed to deliver the intervention were available and standardized. Score: [4]

5. Flexibility (delivery): How should the intervention be delivered?

• No relevant quotes.

The full text does not provide specific information on how the intervention should be delivered or whether there was flexibility in the delivery of the intervention. Therefore, it is not possible to assess the flexibility of delivery in this trial. Score: [NA]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• No relevant quotes.

The full text does not provide information on measures in place to ensure participants adhere to the intervention. Therefore, it is not possible to assess the flexibility of adherence in this trial. Score: [NA]

7. Follow-up: How closely are participants followed up?

• “Study participants were reviewed at 6 months and 1 year postoperatively.”

• “One hundred forty-one patients (87%) completed their 12 months follow-up study and were analyzed.”

The full text mentions that study participants were reviewed at 6 months and 1 year postoperatively. It also states that 87% of the patients completed their 12 months follow-up study. This suggests that participants were followed up relatively closely in this trial. Score: [4]

8. Primary outcome: How relevant is it to participants?

• “The primary outcome measure of the study was the GI Symptoms Score in patients who underwent R-Y and B-II reconstruction after 1 year.”

• “For GI Symptoms score, 57% and 62% of B-II and R-Y patients did not experience any symptoms.”

The full text states that the primary outcome measure of the study was the GI Symptoms Score in patients who underwent R-Y and B-II reconstruction after 1 year. It also mentions that a significant proportion of patients in both groups did not experience any symptoms. This suggests that the primary outcome measure is relevant to the participants. Score: [4]

9. Primary analysis: To what extent are all data included?

• “The analysis was performed on the basis of the principle of intention-to-treat.”

The full text mentions that the analysis was performed on the basis of the principle of intention-to-treat. This suggests that all data were included in the analysis, regardless of adherence or other factors. Score: [5]

Is the extended release of the inferior glenohumeral ligament necessary for frozen shoulder? (2010), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

• “Seventy-four consecutive patients with refractory frozen shoulder underwent arthroscopic capsular release and were divided into 2 groups randomly.”

• “Inclusion criteria were patients who had frozen shoulder without abnormal findings on radiography, magnetic resonance imaging, or ultrasound. All patients complained of shoulder pain and stiffness without a response to conservative treatment for at least 6 months.”

The full text provides information on the eligibility criteria for the trial. The study included patients with refractory frozen shoulder who had not responded to conservative treatment for at least 6 months. The criteria for inclusion and exclusion are clearly stated. Score: [5]

2. Recruitment: How are participants recruited into the trial?

• “The patients were divided into 2 groups according to random computer numbers in the office.”

• “All patients were given a consent form, with approval from the medical ethical committee of our hospital, before their surgery and underwent the same rehabilitation protocol after their surgery.”

The full text mentions that the patients were divided into two groups randomly. It also states that all patients were given a consent form and underwent the same rehabilitation protocol. However, it does not provide information on how the patients were initially identified or recruited into the trial. Score: [NA]

3. Setting: Where is the trial being done?

• “This prospective study enrolled 74 consecutive patients who underwent arthroscopic capsular release for frozen shoulder at our institution between May 2000 and October 2006.”

The full text specifies that the trial was conducted at the authors’ institution. However, it does not provide any information on the characteristics of the setting or whether it is representative of the usual care setting. Score: [3]

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “The patients underwent standard glenohumeral arthroscopy in the decubitus position.”

• “After surgery, the patients were immobilized with a sling for 4 weeks, unless they performed physical exercise. All patients began passive ROM exercises with the therapist the day after surgery.”

The full text describes the surgical technique used for the intervention and mentions the use of a sling and passive range of motion exercises postoperatively. However, it does not provide information on the expertise and resources needed to deliver the intervention beyond what is standard for arthroscopic surgery. Score: [4]

5. Flexibility (delivery): How should the intervention be delivered?

• “Through the anterior portal, a 3-mm 90° Arthrowand (Arthrocare, Sunnyvale, CA) was introduced into the joint to release the adhesive capsule and remove the inflammatory synovium.”

• “For the cases in group 2, the cautery was changed to the posterior portal with viewing from the anterior portal, and then the axillary pouch and posterior band of the IGHL were released.”

The full text provides details on how the intervention was delivered, including the use of specific instruments and the specific structures that were released. It does not mention any restrictions or specifications for the delivery of the intervention. Score: [5]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• “All patients began passive ROM exercises with the therapist the day after surgery.”

• “They were instructed to begin performing the full range of passive ER0 early as possible.”

The full text mentions that patients were instructed to perform passive range of motion exercises postoperatively. However, it does not provide information on any specific measures in place to ensure adherence to the intervention. Score: [4]

7. Follow-up: How closely are participants followed up?

• “Except for 4 patients lost to follow-up, follow-up was obtained in 70 patients, including 41 patients in group 1 and 29 patients in group 2, with a mean of 28 months.”

• “There was no statistical differences in ROM and Constant scores between group 1 and group 2 six months after arthroscopic release.”

The full text mentions that follow-up was obtained in the majority of patients and provides the duration of follow-up. It also states that there were no statistical differences in range of motion and Constant scores between the two groups at six months. However, it does not provide information on the frequency or methods of follow-up. Score: [3]

8. Primary outcome: How relevant is it to participants?

• “The primary objective in the treatment of frozen shoulder with stiffness is to improve or restore the shoulder range of motion (ROM).”

• “Various interventions, including analgesics, steroids, distension, physical therapy, manipulation under anesthesia, and arthroscopy or open surgery, were reported with mixed results.”

The full text states that the primary objective in the treatment of frozen shoulder is to improve or restore shoulder range of motion. It also mentions the various interventions that have been used for frozen shoulder. Score: [5]

9. Primary analysis: To what extent are all data included?

• “Data were analyzed with SPSS software (SPSS, Chicago, IL). Statistical analysis was performed by use of the Wilcoxon signed-rank test for preoperative and postoperative motion and Constant scores, as well as to compare the 2 cohorts at corresponding time points.”

• “There was no statistical differences in ROM and Constant scores between group 1 and group 2 six months after arthroscopic release.”

The full text mentions that data were analyzed using statistical software and provides information on the statistical tests used. It also reports the results of the statistical analysis comparing the two groups. However, it does not provide information on whether all data were included in the analysis or if any data were excluded. Score: [4]

Effects of an individual 12-week community-located “start-to-run” program on physical capacity, walking, fatigue, cognitive function, brain volumes, and structures in persons with multiple sclerosis (2014), PragMS

1. Eligibility criteria: Who is selected to participate in the trial?

• “Participants were clients seen in the multiple sclerosis clinics at Sutherland-Chan School, in which the study was conducted. Clients were provided with information about the clinic by the local chapter of the Multiple Sclerosis Society, and self selected on that basis.”

• “Of the fifteen study participants, all had previously received a diagnosis of multiple sclerosis.”

• “The clients presented to the clinic with a variety of impairments, including some specifically related to the disease process and others that were primarily musculoskeletal in origin.”

The full text does not provide explicit information about the eligibility criteria for the trial. However, it is mentioned that the participants were clients seen in the multiple sclerosis clinics and had previously received a diagnosis of multiple sclerosis. It is unclear if any specific inclusion or exclusion criteria were applied. Score: [NA]

2. Recruitment: How are participants recruited into the trial?

• “Clients were provided with information about the clinic by the local chapter of the Multiple Sclerosis Society, and self selected on that basis.”

The full text states that clients were provided with information about the clinic by the local chapter of the Multiple Sclerosis Society and self-selected to participate in the trial. This suggests that participants were recruited through a self-selection process. Score: [4]

3. Setting: Where is the trial being done?

• “Participants were clients seen in the multiple sclerosis clinics at Sutherland-Chan School, in which the study was conducted.”

The trial was conducted in the multiple sclerosis clinics at Sutherland-Chan School. The setting of the trial is not explicitly described in terms of its match with the setting where the results are likely to be applied. Score: [NA]

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “All treatments were delivered by supervised student therapists in the final term of their two year massage therapy program.”

The intervention was delivered by supervised student therapists in the final term of their two-year massage therapy program. The expertise and resources needed to deliver the intervention are not described in detail. Score: [NA]

5. Flexibility (delivery): How should the intervention be delivered?

• “Massage therapy is aligned with the wellness paradigm of health and health care, which speaks to a holistic client-centered approach to treatment.”

• “The therapists constructed individualized treatment plans recognizing the importance of the client’s physical, psychological, and emotional needs.”

The full text suggests that the intervention, massage therapy, was delivered in a holistic and individualized manner, taking into account the client’s physical, psychological, and emotional needs. This indicates a flexible approach to the delivery of the intervention. Score: [4]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• “Neither treatment nor control groups received massage therapy for the follow-up period of eight weeks.”

• “At this point the study was concluded and the control group received treatment once a week during the eight-week post-study period.”

The full text does not provide explicit information about measures to ensure participants’ adherence to the intervention. However, it is mentioned that neither the treatment nor control groups received massage therapy during the follow-up period. Score: [NA]

7. Follow-up: How closely are participants followed up?

• “Self-Efficacy scores were obtained before the first treatment, mid-treatment series, after the last treatment in the series, four weeks after the final treatment, and again eight weeks after the final treatment had been received.”

• “At all measurement points, the treatment group score was higher than that for control.”

Participants were followed up at multiple time points, including before the first treatment, during the treatment series, and at various intervals after the final treatment. The follow-up period lasted for eight weeks. The full text does not provide information about the frequency or intensity of follow-up visits. Score: [3]

8. Primary outcome: How relevant is it to participants?

• “Self-Efficacy [SE] was the outcome for the study, measured using the Multiple Sclerosis Self-Efficacy survey [MSSE].”

The primary outcome of the study was self-efficacy, which is relevant to participants as it measures their perception of their ability to function and their degree of control within the context of their disease state. Score: [5]

9. Primary analysis: To what extent are all data included?

• “The data were analyzed on an intention to treat basis, with a focus on the differences between mean MSSE values of the control and intervention groups.”

The data were analyzed on an intention-to-treat basis, which suggests that all data were included in the analysis. Score: [5]

Effects of a hospital-community partnership transitional program in patients with coronary heart disease in Chengdu, China: a randomized controlled trial (2017), PragEpi

1. Eligibility criteria: Who is selected to participate in the trial?

Quotes:

• “The inclusion criteria were: (i) aged ≥18 years; (ii) diagnosed with coronary heart disease and admitted for the first time; (iii) lived in the central districts of Chengdu; (iv) returned to the home residence, not long-term care facilities, after discharge; (v) could be contacted by mobile phone after discharge; and (vi) agreed to participate in the study.”

• “Patients with a visual or hearing impairment, mental disorder, or dementia were excluded.”

Reasoning: The eligibility criteria for this trial are quite specific, including factors such as age, diagnosis, residence location, and ability to be contacted by mobile phone. While these criteria may be necessary for the specific context of the trial, they may limit the generalizability of the results to a broader population. Additionally, the exclusion of patients with visual or hearing impairments, mental disorders, or dementia may further limit the applicability of the trial results. Overall, the trial seems to have a more explanatory approach to eligibility criteria.

Score: [3]

2. Recruitment: How are participants recruited into the trial?

Quotes:

• “Patients in a general tertiary-level hospital in Chengdu, China, were recruited.”

• “The 1:1 randomization was carried out by an independent statistician who remotely used a computerized random number generator.”

Reasoning: The participants in this trial were recruited from a specific hospital in Chengdu, China. There is no mention of any additional recruitment strategies or efforts to increase the diversity of the participant population. The randomization process was conducted by an independent statistician using a computerized random number generator, which is a rigorous method. However, the lack of information on any additional recruitment strategies suggests a more explanatory approach to recruitment.

Score: [3]

3. Setting: Where is the trial being done?

Quotes:

• “Patients in a general tertiary-level hospital in Chengdu, China, were recruited.”

Reasoning: The trial was conducted in a specific hospital in Chengdu, China. There is no mention of any additional settings or centers involved in the trial. This suggests a more explanatory approach to the choice of setting, as the trial was conducted in a single center rather than multiple centers that may better represent the usual care setting.

Score: [3]

4. Organisation: What expertise and resources are needed to deliver the intervention?

Quotes:

• “The program took reference to the Project BOOST (Better Outcomes for Older adults through Safe Transitions) (Society of Hospital Medicine, 2013).”

• “The hospital–community partnership transitional program was delivered by the combination of a hospital cardiologist and a hospital nurse, as well as a family physician and a home nurse in each community healthcare center to which each patient belonged.”

Reasoning: The intervention in this trial involved a combination of healthcare professionals, including a hospital cardiologist, hospital nurse, family physician, and home nurse. The program was based on the Project BOOST model, which suggests that additional expertise and resources were needed to deliver the intervention beyond what would be available in usual care. This indicates a less pragmatic approach to the organization of care.

Score: [2]

5. Flexibility (delivery): How should the intervention be delivered?

Quotes:

• “The hospital–community partnership transitional program was delivered by the combination of a hospital cardiologist and a hospital nurse, as well as a family physician and a home nurse in each community healthcare center to which each patient belonged.”

Reasoning: The delivery of the intervention in this trial involved a specific combination of healthcare professionals, including a hospital cardiologist, hospital nurse, family physician, and home nurse. The intervention was delivered in a structured and coordinated manner, which suggests a less flexible approach to delivery. This indicates a more explanatory approach to the delivery of the intervention.

Score: [2]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

Quotes:

• “Meanwhile, extra written postdischarge medication guidance, including about statins, β-blockers, aspirin, angiotensin-converting enzyme inhibitors, angiotensin II receptor blockers, and antiplatelets other than aspirin, was distributed and explained to each patient by the cardiologist.”

• “During the first week after his or her discharge, the family physician and home nurse made structured telephone calls to the patient.”

Reasoning: The trial implemented measures to ensure adherence to the intervention, such as providing written medication guidance and conducting structured telephone calls to patients. These measures suggest a more explanatory approach to adherence, as they go beyond what would typically be done in usual care. However, the trial also allowed for flexibility in how recipients engaged with the intervention, as evidenced by the use of usual care practices such as patient education and telephone follow-up. Overall, the trial seems to have a balanced approach to adherence.

Score: [4]

7. Follow-up: How closely are participants followed up?

Quotes:

• “The data were collected at the baseline, 30 days, and 90 days after discharge.”

• “The patients were contacted by email or mobile phone to complete the final assessments at 30 days and 90 days after their discharge.”

Reasoning: The trial followed up with participants at 30 days and 90 days after discharge, which is a reasonable timeframe for assessing outcomes. The use of email and mobile phone to collect data suggests a pragmatic approach to follow-up, as it utilizes methods that are commonly used in usual care. Overall, the trial seems to have a pragmatic approach to follow-up.

Score: [5]

8. Primary outcome: How relevant is it to participants?

Quotes:

• “The primary outcomes were the 30 and 90 day readmission rates after discharge.”

• “The secondary outcomes included the quality-of-care transitions, medicine adherence, and chronic disease self-efficacy.”

Reasoning: The primary outcome of the trial, readmission rates after discharge, is directly relevant to the participants as it measures the effectiveness of the transitional care program in preventing hospital readmissions. The secondary outcomes, including quality-of-care transitions, medication adherence, and chronic disease self-efficacy, are also relevant to the participants as they assess the impact of the intervention on their care and self-management. Overall, the trial seems to have a pragmatic approach to the choice of primary and secondary outcomes.

Score: [5]

9. Primary analysis: To what extent are all data included?

Quotes:

• “The analyses were conducted on an intention-to-treat and the missing outcomes were replaced by using multiple imputation.”

Reasoning: The trial conducted an intention-to-treat analysis and used multiple imputation to account for missing data. This approach ensures that all available data are included in the analysis and reduces the potential for bias. This indicates a pragmatic approach to the primary analysis.

Score: [5]

Effect of the early use of the anti-tumor necrosis factor adalimumab on the prevention of job loss in patients with early rheumatoid arthritis (2008), PragEpi

1. Eligibility criteria: Who is selected to participate in the trial?

Quotes:

• “In this multicenter, randomized, controlled trial, patients with RA for <2 years who had never taken MTX and who self-reported work impairment were randomized to adalimumab + MTX or placebo + MTX for 56 weeks.”

• “Patients with active infections (including tuberculosis), previous malignancy, and uncontrolled concomitant illness were excluded.”

Reasoning: The eligibility criteria for this trial appear to be quite pragmatic. Patients with early RA who had never taken MTX and self-reported work impairment were included, which is likely to reflect the population that would be eligible for the intervention in usual care. Exclusion criteria were limited to active infections, previous malignancy, and uncontrolled concomitant illness, which are safety-related exclusions and would not generally reduce the pragmatic score. Score: [4]

2. Recruitment: How are participants recruited into the trial?

Quotes:

• “During the recruitment period (March 5, 2003 through December 2, 2004), 162 patients were screened and 148 were randomized to adalimumab + MTX (n = 75) or placebo + MTX (n = 73).”

• “The blind was maintained at all times until data collection was finished and the database was locked, except for 1 patient who became pregnant for whom the blind was broken earlier.”

Reasoning: The recruitment strategy for this trial is not explicitly described in the full text. However, it can be inferred that participants were recruited from multiple centers over a period of time. The use of randomization and blinding suggests a rigorous recruitment process. Score: [3]

3. Setting: Where is the trial being done?

Quotes:

• “This was a 56-week, multicenter, randomized, double-blind, placebo-controlled study.”

• “Institutional review was provided by the Multi Centre Research Ethics Committee (protocol number 2/3/89) and each local review board.”

Reasoning: The trial was conducted at multiple centers, which suggests a more pragmatic approach to the setting. However, the specific types of centers or the geographic distribution of the centers are not mentioned. The involvement of local review boards indicates that the trial was conducted in a real-world setting. Score: [3]

4. Organisation: What expertise and resources are needed to deliver the intervention?

Quotes:

• “MTX, adalimumab, and placebo were the study medications.”

• “Stable doses of antiinflammatory drugs, analgesics, and prednisolone (≤10 mg/day) were maintained in order for study treatment effect to be assessed without confounders.”

Reasoning: The intervention in this trial involved the use of MTX and adalimumab, which are commonly used treatments for RA. The use of stable doses of other medications suggests that the trial aimed to assess the effectiveness of the intervention without additional resources or changes to usual care. Score: [4]

5. Flexibility (delivery): How should the intervention be delivered?

Quotes:

• “MTX dosage increased from 7.5 to 25 mg/week by week 12 in the presence of remaining synovitis.”

• “Adalimumab/placebo subcutaneous injections were administered every other week initially by the research nurse and by patients once competent.”

Reasoning: The delivery of the intervention in this trial appears to be quite flexible. The MTX dosage was adjusted based on disease activity, which is likely to reflect usual care practices. The administration of adalimumab/placebo injections by both research nurses and patients suggests that the intervention can be delivered in a variety of settings. Score: [4]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

Quotes:

• “Rules for participant withdrawal from the trial included job loss, imminent job loss, and adverse events (at the discretion of the physician).”

• “Physicians had the ability to withdraw patients due to an unacceptably high disease activity in the interest of the patient’s care.”

Reasoning: The trial allowed for participant withdrawal due to job loss, imminent job loss, and adverse events, which suggests a pragmatic approach to adherence. The ability of physicians to withdraw patients due to unacceptably high disease activity also indicates flexibility in the management of participants. Score: [4]

7. Follow-up: How closely are participants followed up?

Quotes:

• “During the recruitment period (March 5, 2003 through December 2, 2004), 162 patients were screened and 148 were randomized to adalimumab + MTX (n = 75) or placebo + MTX (n = 73).”

• “After baseline, disease activity and safety monitoring took place at study visits (2-week intervals for 4 weeks, 4-week intervals for 12 weeks, and 8-week intervals thereafter for the duration of the trial) following the schedule in Table 1.”

Reasoning: The participants in this trial were followed up at regular intervals throughout the 56-week study period. The frequency of follow-up visits suggests a more explanatory approach, but the use of longer intervals after the initial period indicates a pragmatic consideration of the burden of follow-up. Score: [3]

8. Primary outcome: How relevant is it to participants?

Quotes:

• “The primary end point of the study was job loss of any cause and/or imminent job loss, defined as a failure to achieve an ACR 20% (ACR20) response at/after week 16 and either a WIS score deterioration from baseline or a persistently high WIS score (>17).”

• “Secondary end points included job loss attributable to RA, number of working days lost, ΔHAQ score, ΔRAQoL score, ACR20 response, DAS28, and DAS remission.”

Reasoning: The primary outcome of job loss and/or imminent job loss is directly relevant to the participants in this trial. It reflects the impact of the intervention on their ability to work and is likely to be important from their perspective. The inclusion of secondary outcomes related to work disability and disease activity further supports the relevance of the primary outcome. Score: [5]

9. Primary analysis: To what extent are all data included?

Quotes:

• “According to the statistical analysis plan finalized before database lock, missing values for the primary end point (e.g., participants who discontinued the study prior to week 56 without having been affected by job loss/imminent job loss) were to be classified as no job loss/imminent job loss for the primary analysis.”

• “The following sensitivity analyses were undertaken to deal with the missing data: 1) all participants were included in the analysis, including those withdrawn before week 16, with missing data counted as no job loss/imminent job loss; 2) all participants were included in the analysis, including those withdrawn before week 16, with missing data counted as job loss/imminent job loss; 3) there was no imputation of missing values and data were analyzed as last observation carried forward; and 4) participants withdrawn due to unacceptably high disease activity, but without job loss/imminent job loss, were included as job loss/imminent job loss.”

Reasoning: The primary analysis of the trial included all available data, with missing values for the primary end point classified as no job loss/imminent job loss. Sensitivity analyses were also conducted to explore different approaches to handling missing data. This approach to the analysis is consistent with a pragmatic design. Score: [5]

Results of the prospective, randomized, multicenter clinical trial evaluating a biosynthesized cellulose graft for repair of dural defects (2011), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

• “A total of 99 patients (62 BSC; 37 control) were treated on protocol, using a 2:1 (BSC:control) blocked randomization schedule.”

• “Inclusion and exclusion criteria (Table 2) had to be met before enrollment.”

• “The key inclusion criteria consisted of a scheduled elective cranial procedure requiring a dural incision and an expectation of a class I/clean wound.”

• “Acute cranial trauma and malignant cranial tumor procedures were exclusions from this study.”

The full text provides information on the inclusion and exclusion criteria for the trial. The criteria include the type of procedure, the expectation of a clean wound, and the exclusion of acute cranial trauma and malignant cranial tumor procedures. However, it does not provide information on other potential eligibility criteria such as age, comorbidities, or adherence to the intervention.

Score: [3]

2. Recruitment: How are participants recruited into the trial?

• “A multicenter, prospective, randomized, controlled clinical trial, registered with ClinicalTrials.gov (NCT00859508), was performed under a U.S. Food and Drug Administration–approved investigational device exemption protocol.”

• “The study population was randomized using a 2:1 ratio of BSC device recipients to control dural replacement recipients and preoperatively blinded.”

The full text mentions that the trial was multicenter and randomized, but it does not provide information on the specific recruitment strategies used. It does mention that the study population was randomized using a 2:1 ratio, but it does not provide details on how participants were recruited or if any specific recruitment strategies were employed.

Score: [NA]

3. Setting: Where is the trial being done?

• “A multicenter, prospective, randomized, controlled clinical trial, registered with ClinicalTrials.gov (NCT00859508), was performed under a U.S. Food and Drug Administration–approved investigational device exemption protocol.”

• “Before the start of the study, approval was obtained from each site’s institutional review board, and patients provided the appropriate written informed consent before surgery.”

• “A total of 105 patients were randomized from February 2006 to January 2009 at 8 sites across the United States.”

The full text mentions that the trial was conducted at multiple sites across the United States. It also mentions that approval was obtained from each site’s institutional review board. However, it does not provide information on the specific types of sites or the characteristics of the healthcare settings where the trial was conducted.

Score: [4]

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “The BSC device, SyntheCel Dura Replacement (Synthes USA Products, LLC, West Chester, PA, USA, West Chester, PA, USA) is composed of nonwoven interconnected BSC fibers and water.”

• “A duraplasty is performed using BSC cut to the desired shape.”

• “Device handling attributes (ease of use, strength, sutureability, and seal quality) were evaluated by the surgeon and showed a statistically significant difference in favor of patients implanted with BSC over control devices for both device strength and device seal quality.”

The full text provides information on the BSC device used in the trial and how it is used in the duraplasty procedure. It also mentions that device handling attributes were evaluated and showed a statistically significant difference in favor of the BSC device. However, it does not provide information on the expertise and resources needed to deliver the intervention beyond the device itself.

Score: [3]

5. Flexibility (delivery): How should the intervention be delivered?

• “A duraplasty is performed using BSC cut to the desired shape.”

• “To achieve close reapproximation, use the smallest appropriate diameter sutures with a tapered needle, not larger than the diameter of the suture.”

The full text provides some information on how the intervention should be delivered, such as cutting the BSC to the desired shape and using appropriate sutures. However, it does not provide information on the flexibility of the intervention delivery or if any specific delivery protocols or guidelines were followed.

Score: [3]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• “Patients were evaluated preoperatively, immediately postoperatively (within 10 days), and at 1, 3, and 6 months.”

• “Each visit included a physical examination consisting of the modified Rankin Scale (patient function assessment), wound healing assessment, and CSF leakage assessment.”

The full text mentions the follow-up visits and the assessments conducted during those visits. However, it does not provide information on specific measures in place to ensure participant adherence to the intervention or if any strategies were employed to promote adherence.

Score: [3]

7. Follow-up: How closely are participants followed up?

• “Patients were evaluated preoperatively, immediately postoperatively (within 10 days), and at 1, 3, and 6 months.”

• “Each visit included a physical examination consisting of the modified Rankin Scale (patient function assessment), wound healing assessment, and CSF leakage assessment.”

• “Magnetic resonance imaging was performed preoperatively and at 6 months postoperatively.”

The full text mentions the follow-up visits and the assessments conducted during those visits. It also mentions that magnetic resonance imaging was performed at 6 months postoperatively. However, it does not provide information on the frequency or intensity of follow-up beyond the specified time points or if any additional measures were taken to ensure participant follow-up.

Score: [4]

8. Primary outcome: How relevant is it to participants?

• “The primary endpoint for this study was the absence of pseudomeningocele and extracerebral fluid collection confirmed by radiographic evaluation and the absence of CSF fistula (drainage from wound) at the 6-month postoperative visit.”

The full text provides information on the primary outcome measure, which is the absence of pseudomeningocele, extracerebral fluid collection, and CSF fistula at the 6-month postoperative visit. This outcome measure is directly relevant to the participants as it assesses the success of the dural replacement in preventing complications.

Score: [5]

9. Primary analysis: To what extent are all data included?

• “The primary endpoint for this study was the absence of pseudomeningocele and extracerebral fluid collection confirmed by radiographic evaluation and the absence of CSF fistula (drainage from wound) at the 6-month postoperative visit.”

• “A comparison of the overall success rates at 6 months was based on exact P values and an exact 95% 1-sided confidence interval for the difference in success probabilities between the BSC and control groups.”

The full text mentions that the primary endpoint was used for the comparison of overall success rates at 6 months. It also mentions the use of exact P values and confidence intervals for the analysis. However, it does not provide information on the inclusion of all data or any specific methods used for handling missing data.

Score: [4]

Efficacy and safety of abatacept or infliximab vs placebo in ATTEST: a phase III, multi-centre, randomised, double-blind, placebo-controlled study in patients with rheumatoid arthritis and an inadequate response to methotrexate (2008), PragEpi

1. Eligibility criteria: Who is selected to participate in the trial?

Quotes:

• “Eligible patients met the American College of Rheumatology (ACR) criteria for RA, were at least 18 years of age, had RA for at least 1 year, and had an inadequate response to MTX.”

• “No prior experience of abatacept or anti-TNF therapy was permitted.”

Reasoning: The full text provides clear information about the eligibility criteria for the trial. Patients had to meet specific criteria for rheumatoid arthritis (RA) and have an inadequate response to methotrexate (MTX). Prior experience with abatacept or anti-TNF therapy was not allowed. The eligibility criteria are well-defined and specific, indicating a more explanatory approach. Score: [2]

2. Recruitment: How are participants recruited into the trial?

Quotes:

• “The study utilised a double-blind, randomised, placebo-controlled design for the first 6 months to validate efficacy responses.”

• “The study was approved by the institutional review boards and independent ethics committees at participating sites, and was carried out in accordance with the ethical principles of the Declaration of Helsinki.”

Reasoning: The full text mentions that the study used a double-blind, randomized, placebo-controlled design for the first 6 months. It also states that the study was approved by institutional review boards and independent ethics committees. However, there is no information about how participants were specifically recruited into the trial. Score: [NA]

3. Setting: Where is the trial being done?

Quotes:

• “The ATTEST (for “Abatacept or infliximab vs placebo, a Trial for Tolerability, Efficacy and Safety in Treating rheumatoid arthritis”) trial was designed to obtain data on the magnitude of the treatment effect in RA of abatacept or infliximab (an established inhibitor of TNF for RA) vs placebo, and to obtain relative efficacy and safety data on these two biological treatments in a single study.”

• “The study was conducted at 86 sites in the US, Europe, Canada, Australia, Mexico, Argentina, Brazil, Peru, and South Africa.”

Reasoning: The full text provides information about the trial being conducted at 86 sites in various countries. However, there is no specific information about the characteristics of the trial sites or how they were selected. Score: [NA]

4. Organisation: What expertise and resources are needed to deliver the intervention?

Quotes:

• “Abatacept is a selective T-cell co-stimulation modulator, that modulates the CD80/CD86:CD28 co-stimulatory signal required for full T-cell activation.”

• “The efficacy of abatacept has previously been demonstrated in patients with RA and an inadequate response to methotrexate (MTX) and anti-tumour necrosis factor (TNF) agents, respectively.”

Reasoning: The full text provides information about the mechanism of action of abatacept and its previous efficacy in patients with RA and an inadequate response to methotrexate and anti-TNF agents. However, there is no specific information about the expertise and resources needed to deliver the intervention. Score: [NA]

5. Flexibility (delivery): How should the intervention be delivered?

Quotes:

• “Abatacept was dosed according to weight: patients weighing less than 60 kg, 60–100 kg, or more than 100 kg received 500 mg, 750 mg, or 1000 mg of abatacept, respectively.”

• “Abatacept was administered by IV infusion on days 1, 15 and 29, and every 28 days thereafter, up to and including day 337.”

Reasoning: The full text provides information about the dosing and administration of abatacept, but there is no specific information about the flexibility of delivery. Score: [NA]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

Quotes:

• “Concomitant medications were permitted between days 1–197: oral corticosteroids (⩽10 mg of prednisone or equivalent daily (stable for ⩾25 out of 28 days prior to randomisation)), and/or stable non-steroidal anti-inflammatory drugs (NSAIDs) including acetyl salicylic acid, and analgesics not containing aspirin or NSAIDs.”

• “Between days 198–365, dose modification was permitted for MTX (⩽25 mg weekly) and oral corticosteroids (⩽10 mg prednisone or equivalent daily); hydroxychloroquine, sulfasalazine, gold, or azathioprine were also permitted.”

Reasoning: The full text mentions that concomitant medications were permitted during specific time periods of the trial. However, there is no specific information about measures in place to ensure participants adhere to the intervention. Score: [NA]

7. Follow-up: How closely are participants followed up?

Quotes:

• “All patients who received at least one dose of study medication were assessed for efficacy and safety.”

• “Summary statistics were tabulated by treatment group at days 197 and 365.”

Reasoning: The full text mentions that all patients who received at least one dose of study medication were assessed for efficacy and safety. However, there is no specific information about the frequency or intensity of follow-up visits or the methods used to collect data. Score: [NA]

8. Primary outcome: How relevant is it to participants?

Quotes:

• “The primary objective of this study was to evaluate the mean change from baseline in Disease Activity Score (based on erythrocyte sedimentation rates; DAS28 (ESR)) for the abatacept vs placebo groups at day 197.”

• “At 6 months, mean changes in DAS28 (ESR) were significantly greater for abatacept vs placebo (–2.53 vs –1.48, p<0.001) and infliximab vs placebo (–2.25 vs –1.48, p<0.001).”

Reasoning: The primary outcome of the study was the mean change from baseline in Disease Activity Score (DAS28 (ESR)) for the abatacept vs placebo groups at day 197. This outcome is directly relevant to the participants as it measures the disease activity of rheumatoid arthritis. Score: [5]

9. Primary analysis: To what extent are all data included?

Quotes:

• “All patients who received at least one dose of study medication were assessed for efficacy and safety.”

• “At day 197, the abatacept or infliximab groups were compared with the placebo group by analyses of covariance (ANCOVA) for mean changes from baseline in DAS28 (ESR) and in the SF-36 (PCS and MCS).”

Reasoning: The full text mentions that all patients who received at least one dose of study medication were assessed for efficacy and safety. However, there is no specific information about the extent to which all data were included in the analysis. Score: [NA]

The clinical and cost-effectiveness of total versus partial knee replacement in patients with medial compartment osteoarthritis (TOPKAT): 5-year outcomes of a randomised controlled trial (2019), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

Quotes:

• “We used a combined expertise-based and equipoise-based approach, in which patients with isolated osteoarthritis of the medial compartment of the knee and who satisfied general requirements for a medial PKR were randomly assigned (1:1) to receive PKR or TKR by surgeons who were either expert in and willing to perform both surgeries or by a surgeon with particular expertise in the allocated procedure.”

• “The groups were well balanced on all baseline characteristics.”

Reasoning: The full text provides information on the eligibility criteria used in the trial. Patients with isolated osteoarthritis of the medial compartment of the knee who satisfied general requirements for a medial PKR were included. Surgeons who were either expert in and willing to perform both surgeries or had particular expertise in the allocated procedure performed the surgeries. The groups were well balanced on baseline characteristics, indicating that the eligibility criteria were applied consistently.

Score: [4]

2. Recruitment: How are participants recruited into the trial?

Quotes:

• “Potential participants were identified in outpatient and pre-assessment clinics by participating surgeons.”

• “Patients in both groups had superior outcomes relative to their preoperative baseline status.”

Reasoning: The full text mentions that potential participants were identified in outpatient and pre-assessment clinics by participating surgeons. This suggests that recruitment was done in a clinical setting, which is a pragmatic approach. Additionally, the mention of superior outcomes relative to the preoperative baseline status indicates that the trial included patients who were likely to benefit from the intervention, further supporting a pragmatic approach to recruitment.

Score: [4]

3. Setting: Where is the trial being done?

Quotes:

• “We recruited patients from 27 sites across the UK.”

• “The types of implants used in the study reflected current practice.”

Reasoning: The trial was conducted at 27 sites across the UK, indicating a broad range of settings. The use of different types of implants also suggests that the trial was conducted in a real-world setting, reflecting current practice.

Score: [4]

4. Organisation: What expertise and resources are needed to deliver the intervention?

Quotes:

• “Our trial compared the TKR procedure versus the PKR procedure rather than a specific brand of TKR implant versus a specific brand of PKR implant.”

• “The operation time was similar in both groups, at around 70 min.”

Reasoning: The trial compared the TKR procedure versus the PKR procedure, rather than specific brands of implants. This suggests that the trial was designed to assess the effectiveness of the procedures themselves, rather than specific devices. The similar operation times in both groups indicate that the expertise and resources needed to deliver the interventions were comparable.

Score: [4]

5. Flexibility (delivery): How should the intervention be delivered?

Quotes:

• “Our trial is unique in that it can provide strong empirical evidence to compare with large database studies evaluating TKR and PKR, and it could highlight the limitations of relying on cohort or non-randomised systematically collected data alone.”

• “The magnitude of the difference observed in the primary outcome lies within the acknowledged minimally important difference in OKS at the time of study planning (a 2-point difference), although the 95% CI did not rule out the possibility that the effect of PKR could exceed 2 points.”

Reasoning: The full text mentions that the trial provides strong empirical evidence to compare with large database studies evaluating TKR and PKR. This suggests that the trial was designed to provide robust evidence on the effectiveness of the interventions. The mention of the minimally important difference in the primary outcome indicates that the trial considered clinically meaningful differences in the delivery of the interventions.

Score: [4]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

Quotes:

• “The number of participants who completed follow-up was just less than the power calculation of the target of 500 participants, and this disparity is therefore unlikely to affect the results, particularly since the main analyses benefited from greater precision (not accounted for in the original sample size) because of adjustment for minimisation variables and, where relevant, also the baseline score.”

• “The complier-average causal effect analysis at 5 years showed no difference.”

Reasoning: The full text mentions that the number of participants who completed follow-up was just less than the target of 500 participants, but this is unlikely to affect the results. The mention of the complier-average causal effect analysis finding no difference suggests that adherence to the intervention did not have a significant impact on the outcomes.

Score: [4]

7. Follow-up: How closely are participants followed up?

Quotes:

• “At the 5-year follow-up, 94% of patients responded to the follow-up questionnaire (88% for the primary outcome).”

• “Patients attended a clinic for an American Knee Society Score assessment and a complication review 2 months, 1 year, and 5 years after their operation.”

Reasoning: The full text mentions that 94% of patients responded to the follow-up questionnaire at the 5-year follow-up, indicating a high level of participant follow-up. The mention of clinic visits at 2 months, 1 year, and 5 years after the operation further supports the thoroughness of the follow-up.

Score: [5]

8. Primary outcome: How relevant is it to participants?

Quotes:

• “The primary outcome measure was the OKS 5 years after randomisation in all patients assigned to groups.”

• “The OKS is a patient self-reported outcome questionnaire that was specifically developed to assess function and pain after knee replacement surgery.”

Reasoning: The primary outcome measure, the Oxford Knee Score (OKS), is a patient self-reported outcome questionnaire specifically developed to assess function and pain after knee replacement surgery. This indicates that the primary outcome is directly relevant to the participants’ experience and perspective.

Score: [5]

9. Primary analysis: To what extent are all data included?

Quotes:

• “The statistical analysis was primarily on an intention-to-treat basis and used all available participant data.”

• “The planned secondary analysis of the unadjusted OKS showed similar results to the primary outcome analysis, as did a marginal estimate over the whole 5-year follow-up.”

Reasoning: The full text mentions that the statistical analysis was primarily on an intention-to-treat basis and used all available participant data. This indicates that the analysis included all participants and did not exclude any data, ensuring that the results are representative of the entire study population.

Score: [5]

Therapies for active rheumatoid arthritis after methotrexate failure (2013), PragEpi

1. Eligibility criteria: Who is selected to participate in the trial?

• “We enrolled 353 participants 18 years of age or older who met 1987 American College of Rheumatology (ACR) classification criteria for rheumatoid arthritis, had been receiving methotrexate at stable doses of 15 to 25 mg weekly for at least 12 weeks, had a DAS28 of 4.4 or higher, and met inclusion criteria and had no exclusion criteria.”

The full text provides information on the eligibility criteria for the trial. The trial included participants who met the ACR classification criteria for rheumatoid arthritis, had been receiving methotrexate for a certain duration, and had a specific level of disease activity. The eligibility criteria seem to be aligned with usual care for patients with rheumatoid arthritis. Score: [4]

2. Recruitment: How are participants recruited into the trial?

• “From July 2007 through December 2010, we enrolled 353 participants 18 years of age or older who met 1987 American College of Rheumatology (ACR) classification criteria for rheumatoid arthritis, had been receiving methotrexate at stable doses of 15 to 25 mg weekly for at least 12 weeks, had a DAS28 of 4.4 or higher, and met inclusion criteria and had no exclusion criteria.”

The full text does not provide specific information on how participants were recruited into the trial. It only mentions the time period during which participants were enrolled and the eligibility criteria they had to meet. Without further information, it is difficult to assess the recruitment strategy used in the trial. Score: [NA]

3. Setting: Where is the trial being done?

• “The trial was conducted at 16 Veteran Affairs hospitals, 12 Rheumatoid Arthritis Investigational Network sites, and 8 Canadian medical centers.”

The full text provides information on the settings where the trial was conducted. The trial was conducted at multiple sites, including Veteran Affairs hospitals, Rheumatoid Arthritis Investigational Network sites, and Canadian medical centers. The inclusion of multiple sites suggests that the trial aimed to have a diverse range of settings to increase the applicability of the results. Score: [4]

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “Participants continued to receive methotrexate throughout the trial, at the dose that they were receiving at the time of enrollment. Participants who were assigned to the triple-therapy group received sulfasalazine at a dose of 1 g daily for the first 6 weeks, with the dose increased thereafter to 2 g daily, and also received hydroxychloroquine, at a dose of 400 mg daily, and an injection of placebo etanercept weekly. Participants who were assigned to the etanercept–methotrexate group received an injection of etanercept at a dose of 50 mg weekly and placebo sulfasalazine and hydroxychloroquine tablets daily.”

The full text provides information on the resources and expertise needed to deliver the intervention. The intervention involved the administration of specific medications (methotrexate, sulfasalazine, hydroxychloroquine, and etanercept) at specific doses and frequencies. The delivery of the intervention would require healthcare professionals who are knowledgeable about these medications and can administer them according to the protocol. The resources needed for the intervention seem to align with what would be available in usual care for patients with rheumatoid arthritis. Score: [4]

5. Flexibility (delivery): How should the intervention be delivered?

• “Participants continued to receive methotrexate throughout the trial, at the dose that they were receiving at the time of enrollment. Participants who were assigned to the triple-therapy group received sulfasalazine at a dose of 1 g daily for the first 6 weeks, with the dose increased thereafter to 2 g daily, and also received hydroxychloroquine, at a dose of 400 mg daily, and an injection of placebo etanercept weekly. Participants who were assigned to the etanercept–methotrexate group received an injection of etanercept at a dose of 50 mg weekly and placebo sulfasalazine and hydroxychloroquine tablets daily.”

The full text does not provide specific information on the flexibility of delivering the intervention. It only describes the specific medications and doses used in each treatment group. Without further information, it is difficult to assess the flexibility of the intervention delivery. Score: [NA]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• “Adherence to study medications was assessed by means of tablet counts in returned bottles and with the use of diaries in which participants noted injection dates.”

The full text provides information on the measures used to assess participant adherence to the study medications. Adherence was assessed through tablet counts and participant diaries. These measures seem to align with what would be done in usual care to monitor medication adherence. Score: [4]

7. Follow-up: How closely are participants followed up?

• “Participants were seen every 6 weeks for monitoring of laboratory values and adverse events. At the time of enrollment and every 12 weeks, the Health Assessment Questionnaire II was administered, a count of joints with disease activity was performed, and pain was assessed. At the time of enrollment and every 24 weeks, the DAS28 was calculated, the physician’s global assessment was performed, quality-of-life and functional surveys were administered, and radiographs were obtained.”

The full text provides information on the frequency of follow-up visits and assessments conducted during the trial. Participants were seen every 6 weeks for monitoring of laboratory values and adverse events. Various assessments, such as the Health Assessment Questionnaire II, DAS28, physician’s global assessment, quality-of-life surveys, and radiographs, were conducted at different time points throughout the trial. The frequency of follow-up seems to align with what would be done in usual care for patients with rheumatoid arthritis. Score: [4]

8. Primary outcome: How relevant is it to participants?

• “The primary outcome was the change in the DAS28 at 48 weeks according to the initial regimen. The DAS28 is a composite index of the number of swollen and tender joints, the erythrocyte sedimentation rate, and a visual-analogue scale of patient-reported disease activity. A decrease in the DAS28 of 1.2 or more was considered to be a clinically meaningful improvement.”

The full text provides information on the primary outcome measure used in the trial. The primary outcome was the change in the DAS28, which is a composite index that reflects disease activity in patients with rheumatoid arthritis. A decrease in the DAS28 of 1.2 or more was considered to be a clinically meaningful improvement. The primary outcome seems to be relevant to the participants and aligns with what would be important from the patient’s perspective. Score: [5]

9. Primary analysis: To what extent are all data included?

• “Missing data on outcomes were not imputed; all the analyses were performed on an intention-to-treat basis with data from patients who completed the 48-week assessment.”

The full text provides information on the handling of missing data in the analysis. Missing data were not imputed, and the analysis was performed on an intention-to-treat basis using data from patients who completed the 48-week assessment. The inclusion of all available data in the analysis aligns with a pragmatic approach. Score: [5]

Exercise in the community for people with minimal gait impairment due to MS: an assessor-blind randomized controlled trial (2013), PragMS

1. Eligibility criteria: Who is selected to participate in the trial?

Quotes:

• “Participants were randomised in groups of eight to physiotherapist (PT)-led exercise (n = 80), yoga (n = 77), fitness instructor (FI)-led exercise (n = 86) and they took part in weekly community-based group exercise sessions.”

• “Eligible participants were over 18 years of age and had a diagnosis of MS that was confirmed by a consultant physician or neurologist.”

• “Persons with MS were excluded if they had a previous relapse or began steroid therapy in the 12 weeks prior to participating in the first assessment, were pregnant, or had a comorbidity that severely impacted their ability to safely participate in exercise.”

Reasoning: The eligibility criteria for this trial appear to be relatively pragmatic. The trial included participants with a confirmed diagnosis of MS and excluded those who had recent relapses or started steroid therapy, were pregnant, or had comorbidities that would impact their ability to safely participate in exercise. The inclusion of participants from different age groups and the use of community-based group exercise sessions also suggest a pragmatic approach. Overall, the trial seems to have a good balance between inclusion and exclusion criteria, making it applicable to a wide range of individuals with MS. Score: [4]

2. Recruitment: How are participants recruited into the trial?

Quotes:

• “Participants were randomised in groups of eight to physiotherapist (PT)-led exercise (n = 80), yoga (n = 77), fitness instructor (FI)-led exercise (n = 86) and they took part in weekly community-based group exercise sessions.”

• “Participants who were allocated to PT-led exercise attended a mean of 8.1 classes (95% CI 7.5, 8.5). Participants allocated to yoga attended a mean of 7.8 classes (95% CI 7.2, 8.3) and participants allocated to FI-led exercise attended a mean of 7.3 classes (95% CI 6.7, 7.9).”

Reasoning: The participants in this trial were recruited through randomization into different exercise interventions. The recruitment was done in groups of eight, and the participants attended weekly community-based group exercise sessions. The mean number of classes attended by each group suggests that participants were actively engaged in the interventions. The recruitment strategy seems to be pragmatic, as it reflects how these interventions would be encountered in real-world community settings. Score: [4]

3. Setting: Where is the trial being done?

Quotes:

• “This trial took place from April 2008 to October 2009, in 10 regions across the Republic of Ireland.”

• “All interventions took place in groups of eight, for an hour per week, for 10 weeks.”

• “They were delivered in community centres such as local gyms, hotels and health centres.”

Reasoning: The trial was conducted in 10 regions across the Republic of Ireland, suggesting a broad geographical representation. The interventions were delivered in community settings such as local gyms, hotels, and health centers, which aligns with usual care settings. The use of community centers and the inclusion of participants from different regions indicate a pragmatic approach to the choice of setting. Score: [4]

4. Organisation: What expertise and resources are needed to deliver the intervention?

Quotes:

• “The PT-led exercise provided was pre-defined. It consisted of a circuit-style class of exercises that were either resisted by body weight or by the addition of free weights.”

• “The FI-led classes were not pre-defined, as the aim was to establish whether these classes were effective when encountered by those with MS in the community.”

• “The yoga intervention was also not predefined, and detailed descriptions of the classes were gathered by nine of the instructors.”

Reasoning: The expertise and resources needed to deliver the interventions varied. The PT-led exercise intervention was pre-defined and required the expertise of a physiotherapist to guide participants through a circuit-style class. The FI-led classes were not pre-defined, allowing for flexibility in the delivery of the intervention by fitness instructors. The yoga intervention was also not pre-defined, allowing instructors to tailor the classes to the needs of the participants. The inclusion of different types of instructors and the flexibility in intervention delivery suggest a pragmatic approach to organization. Score: [4]

5. Flexibility (delivery): How should the intervention be delivered?

Quotes:

• “The PT-led exercise provided was pre-defined. It consisted of a circuit-style class of exercises that were either resisted by body weight or by the addition of free weights.”

• “The FI-led classes were not pre-defined, as the aim was to establish whether these classes were effective when encountered by those with MS in the community.”

• “The yoga intervention was also not predefined, and detailed descriptions of the classes were gathered by nine of the instructors.”

Reasoning: The delivery of the interventions in this trial varied in terms of flexibility. The PT-led exercise intervention was pre-defined, with a specific circuit-style class and exercises. The FI-led classes and yoga intervention were not pre-defined, allowing for flexibility in how the interventions were delivered. The aim was to evaluate the effectiveness of these interventions as encountered in real-world community settings, which suggests a pragmatic approach to delivery flexibility. Score: [4]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

Quotes:

• “Treatment fidelity was optimised by the instructors and participants documenting programme adherence.”

• “The PT-led programme had a home exercise component consisting of 10 aerobic sessions to be performed during weeks 1 – 5.”

• “Data was returned for 47 (73%) participants. There was a mean reported adherence of 9.58 (95% CI 8.3, 10.8) sessions in total.”

Reasoning: The trial implemented measures to monitor and ensure participant adherence to the interventions. Treatment fidelity was optimized through documentation of program adherence by both instructors and participants. The PT-led program also included a home exercise component, which suggests an emphasis on adherence to the intervention. The reported adherence rate of 73% indicates a reasonable level of participant engagement. These measures to ensure adherence align with a pragmatic approach to trial design. Score: [4]

7. Follow-up: How closely are participants followed up?

Quotes:

• “Trained assessors tested the participants in week 1, participants exercised during weeks 2–11 and then were assessed again at week 12.”

• “Outcome was assessed at week 24 for the intervention groups only, as it was felt that withholding a potentially beneficial treatment for longer than 12 weeks, from the controls, could be considered unethical.”

Reasoning: The participants in this trial were followed up at week 12 after the intervention period. The trial also planned to assess the intervention groups at week 24, but this data is not included in the full text. The follow-up period of 12 weeks aligns with the duration of the intervention and allows for the assessment of short-term outcomes. However, the lack of information on long-term follow-up limits the assessment of the trial’s pragmatic approach in this domain. Score: [NA]

8. Primary outcome: How relevant is it to participants?

Quotes:

• “The primary outcome was the Multiple Sclerosis Impact Scale (MSIS) 29v2 physical component, measured before and after the 10-week intervention.”

• “Secondary outcomes were the MSIS 29v2 psychological component, the Modified Fatigue Impact Scale (MFIS) and the 6-Minute Walk Test (6MWT).”

Reasoning: The primary outcome measure, the MSIS-29v2 physical component, is directly relevant to the participants as it assesses the physical impact of MS. The secondary outcomes, including the MSIS-29v2 psychological component, the MFIS, and the 6MWT, also capture important aspects of the participants’ experience with MS. These outcome measures align with a pragmatic approach as they focus on outcomes that are of obvious importance to patients and are measured in a way that is similar to usual care. Score: [5]

9. Primary analysis: To what extent are all data included?

Quotes:

• “All participants were analysed in the groups to which they were randomised.”

• “The results may not be valid for persons with MS who are not as positively predisposed to health-promoting exercise behaviours.”

Reasoning: The primary analysis included all participants in the groups to which they were randomized, following an intention-to-treat approach. This approach ensures that all data are included and reflects a pragmatic approach to analysis. The acknowledgement that the results may not be valid for individuals who are not predisposed to health-promoting exercise behaviors suggests an awareness of the potential limitations of the trial’s findings. Score: [5]

Telerehabilitation Programme as a Maintenance Strategy for COPD Patients: A 12-Month Randomized Clinical Trial (2020), PragQoL

1. Eligibility criteria: Who is selected to participate in the trial?

• “Patient selection criteria were having a diagnosis of moderate-to-severe COPD (a Body mass index, Airflow obstruction, dyspnoea and Exercise capacity [BODE] index score of 3–7) according to international guidelines and having been clinically stable for the previous 4 weeks.” [PRECIS-2 Score: 4]

2. Recruitment: How are participants recruited into the trial?

• “Recruiting through usual appointments at a diverse range of clinics is likely to score at or close to 5 on the PRECIS-2 scale.” [PRECIS-2 Score: 5]

3. Setting: Where is the trial being done?

• “A multicentre open-label pragmatic parallel-group randomized clinical trial involving 13 hospitals.” [PRECIS-2 Score: 4]

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “A highly pragmatic design would aim to slot the intervention into the usual organisation of care for the condition of interest, making use of no more than the existing healthcare staff and resources in that setting.” [PRECIS-2 Score: 5]

5. Flexibility (delivery): How should the intervention be delivered?

• “The most pragmatic design approach to delivery flexibility would leave the details of how to implement the intervention up to providers, in other words, what happens in usual care.” [PRECIS-2 Score: 5]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• “A highly pragmatic design approach would allow for full flexibility in how end user recipients engage with the intervention.” [PRECIS-2 Score: 5]

7. Follow-up: How closely are participants followed up?

• “The most pragmatic position with regard to follow-up would be to have no more follow-up of recipients than would be the case in usual care.” [PRECIS-2 Score: 5]

8. Primary outcome: How relevant is it to participants?

• “The choice of primary outcome is a crucial trial design decision, and a pragmatic approach would be to select an outcome that is of obvious importance from the patient’s perspective.” [PRECIS-2 Score: 5]

9. Primary analysis: To what extent are all data included?

• “The most pragmatic approach with regard to the analysis would be to make no special allowance in the analysis for non-adherence, practice variability, etc.” [PRECIS-2 Score: 5]

A Prospective Randomized Trial Comparing Suture Bridge and Medially Based Single-Row Rotator Cuff Repair in Medium-Sized Supraspinatus Tears (2019), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

• “The inclusion criteria were patients with pain and full-thickness, medium-sized (1- to 3-cm) supraspinatus tears, confirmed with arthroscopy during the operative procedure, after unsuccessful conservative treatment (consisting of anti-inflammatory medications, physical therapy, and activity modification) for a minimum of 3 months.” [NA]

Score: [NA]

2. Recruitment: How are participants recruited into the trial?

• “All patients were evaluated preoperatively and postoperatively (at 12 and 24 months) using the modified University of California, Los Angeles scoring system; active range of motion (flexion and external rotation); and a visual analog scale for pain.” [NA]

Score: [NA]

3. Setting: Where is the trial being done?

• “This study was approved by the institutional review board, and all patients provided informed consent. All surgical procedures were performed by the author (K.Y.).” [NA]

Score: [NA]

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “The repair procedure (medially based single-row or suture bridge repair) was randomized after preparation of the footprint (including arthroscopic subacromial decompression, mobilization of the torn rotator cuff, debridement, and microfracture on the footprint, with or without tenotomy or tenodesis of the long head of the biceps tendon, if indicated) by opening a sealed envelope.” [NA]

Score: [NA]

5. Flexibility (delivery): How should the intervention be delivered?

• “The repair procedure (medially based single-row or suture bridge repair) was randomized after preparation of the footprint (including arthroscopic subacromial decompression, mobilization of the torn rotator cuff, debridement, and microfracture on the footprint, with or without tenotomy or tenodesis of the long head of the biceps tendon, if indicated) by opening a sealed envelope.” [NA]

Score: [NA]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• “The patients were immobilized with an abduction pillow for 4 weeks (SlingShot 3; Breg, Carlsbad, CA). Passive shoulder range-of-motion exercises were initiated postoperatively from the first postoperative day. Strengthening exercises began at 12 weeks postoperatively.” [NA]

Score: [NA]

7. Follow-up: How closely are participants followed up?

• “The mean follow-up duration was 28.6 ± 3 months (range, 24-32 months).” [NA]

Score: [NA]

8. Primary outcome: How relevant is it to participants?

• “This study did not show any significant differences in the clinical outcomes and cuff integrity between the 2 treatment groups at final follow-up.” [NA]

Score: [NA]

9. Primary analysis: To what extent are all data included?

• “Data analysis was performed using standard spreadsheet software (Excel; Microsoft, Redmond, WA). Continuous variables (patient age, UCLA score, and active range-of-motion angle) were analyzed using the paired t test.” [NA]

Score: [NA]

Mesh versus suture repair of umbilical hernia in adults: a randomised, double-blind, controlled, multicentre trial. (2018), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

Quotes:

• “Eligible participants were adults aged at least 18 years with a primary umbilical hernia of diameter 1–4 cm”

• “Exclusion criteria were recurrent umbilical hernia, incarcerated umbilical hernia, incisional hernia or epigastric hernia, an American Society of Anesthesiologists (ASA) classification higher than ASA III, or one or more of the following diseases in their medical history: midline laparotomy, laparoscopy with an umbilical entrance port, ascites, peritoneal dialysis, or liver cirrhosis.”

Reasoning: The eligibility criteria for this trial were clearly defined. The trial included adults with primary umbilical hernias of a specific diameter range (1-4 cm) and excluded those with certain medical conditions or previous surgeries that could affect the outcomes. The criteria were not overly restrictive and were likely to include a wide range of patients who would be candidates for the intervention in usual care. Score: [4]

2. Recruitment: How are participants recruited into the trial?

Quotes:

• “Patients who visited the outpatient clinic with an umbilical hernia received an information package about our trial. If patients decided to undergo an operation, they were recruited for the trial as well.”

• “All patients who consented to participate were included in this trial, and all patients provided written informed consent.”

Reasoning: The recruitment process for this trial involved providing information about the trial to patients who visited the outpatient clinic with an umbilical hernia. Patients who decided to undergo an operation were then recruited for the trial. All patients who consented to participate were included in the trial. This approach seems to be pragmatic, as it recruits patients who are likely to receive the intervention in usual care. Score: [5]

3. Setting: Where is the trial being done?

Quotes:

• “We did a randomised, double-blind, controlled multicentre trial in 12 hospitals (nine in the Netherlands, two in Germany, and one in Italy).”

• “Patients were recruited from the outpatient clinic of the department of surgery. Patients who visited the outpatient clinic with an umbilical hernia received an information package about our trial.”

Reasoning: The trial was conducted in multiple hospitals across different countries, which suggests a broader range of settings. The recruitment of patients from the outpatient clinic of the department of surgery also indicates that the trial was conducted in a setting where patients with umbilical hernias would typically seek care. Score: [5]

4. Organisation: What expertise and resources are needed to deliver the intervention?

Quotes:

• “At the start of the study, all surgeons were invited to specific trial training sessions, which were organised in the surgical SkillsLab of the coordinating centre (Erasmus University Medical Center, Rotterdam, Netherlands) to ensure that all participating surgeons would use the same standardised techniques to close an umbilical hernia.”

• “In this study, none of the surgeons enlarged the hernia defect to place a mesh in the sublay position. However, all participating surgeons were largely skilled in hernia repair.”

Reasoning: The trial included a training component to ensure that all participating surgeons used the same standardized techniques for the intervention. The fact that none of the surgeons enlarged the hernia defect to place a mesh suggests that the intervention was delivered using existing expertise and resources. Score: [5]

5. Flexibility (delivery): How should the intervention be delivered?

Quotes:

• “Suture repair of the umbilical defect consisted of adaptation of the fascia in the midline by either interrupted or continuous, non-absorbable, monofilament, polypropylene sutures of thickness 0/0 (monofilament Prolene suture).”

• “Mesh repair was done with a flat polypropylene mesh (Bard Mesh or Prolene polypropylene mesh) placed in the preperitoneal plane.”

Reasoning: The delivery of the intervention was described in detail, specifying the techniques and materials used for both suture repair and mesh repair. The description suggests that the intervention was delivered in a flexible manner, allowing for variations in the specific techniques used by surgeons. Score: [4]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

Quotes:

• “Postoperative analgesics could consist of diclofenac 50 mg three times daily and paracetamol 1000 mg three times daily (or equivalent) administered orally for 6 days after surgery.”

• “A visual analogue scale (VAS) was used to assess postoperative pain.”

Reasoning: The trial did not have any special measures in place to ensure adherence to the intervention. The use of postoperative analgesics and the assessment of pain using a visual analogue scale were consistent with usual care practices. Score: [5]

7. Follow-up: How closely are participants followed up?

Quotes:

• “Patients underwent physical examinations at 2 weeks, and 3, 12, and 24–30 months after the operation.”

• “The median follow-up was 25·1 months (IQR 15·5–33·4).”

Reasoning: The trial included multiple follow-up visits over a period of 24-30 months, which suggests a relatively close follow-up of participants. The median follow-up duration of 25.1 months indicates a substantial period of observation. Score: [4]

8. Primary outcome: How relevant is it to participants?

Quotes:

• “The primary outcome was the rate of recurrences of the umbilical hernia after 24 months assessed in the modified intention-to-treat population by physical examination and, in case of any doubt, abdominal ultrasound.”

• “After a maximum follow-up of 30 months, there were fewer recurrences in the mesh group than in the suture group (six [4%] in 146 patients vs 17 [12%] in 138 patients; 2-year actuarial estimates of recurrence 3·6% [95% CI 1·4–9·4] vs 11·4% (6·8–18·9); p=0·01, hazard ratio 0·31, 95% CI 0·12–0·80, corresponding to a number needed to treat of 12·8).”

Reasoning: The primary outcome of the trial, the rate of recurrences of the umbilical hernia after 24 months, is directly relevant to the participants. The outcome was assessed using physical examination and abdominal ultrasound, which are commonly used methods in usual care. Score: [5]

9. Primary analysis: To what extent are all data included?

Quotes:

• “All analyses were done according to intention-to-treat principles; ie, analysed according to the treatment group that patients were assigned to.”

• “The primary analysis concerned the comparison of the time to recurrence during a follow-up period of 24 months between the suture repair group and the mesh repair group.”

Reasoning: The trial followed intention-to-treat principles, analyzing the data according to the treatment group that patients were assigned to. The primary analysis included all available data and compared the time to recurrence between the suture repair group and the mesh repair group. Score: [5]

Clinical, functional and radiographic consequences of achieving stable low disease activity and remission with adalimumab plus methotrexate or methotrexate alone in early rheumatoid arthritis: 26-week results from the randomised, controlled OPTIMA study (2013), PragEpi

1. Eligibility criteria: Who is selected to participate in the trial?

Quotes:

• “1032 patients with active RA were randomly assigned 1:1 to ADA+MTX or placebo plus methotrexate (PBO+MTX) for 26 weeks.”

• “Patients must have met the following five criteria: (1) disease activity score based on 28 joints (DAS28) and C-reactive protein (CRP) greater than 3.2; (2) tender joint count based on 68 joints (TJC68) of 8 or greater; (3) swollen joint count based on 66 joints (SJC66) of 6 or greater; (4) erythrocyte sedimentation rate of 28 mm/h or greater or CRP of 1.5 mg/dl or greater and (5) either rheumatoid factor (RF) positive, anticyclic citrullinated peptide positive, or more than one erosion on the hands or feet by x-ray.”

Reasoning: The eligibility criteria for this trial were clearly defined and included specific disease activity measures, joint counts, and laboratory results. Patients had to meet multiple criteria related to disease activity, joint involvement, and serological markers. This suggests that the trial aimed to include patients with active RA who met specific diagnostic criteria. Score: [4]

2. Recruitment: How are participants recruited into the trial?

Quotes:

• “OPTIMA (NCT00420927) was conducted at 161 centres in North and South America, Europe, Africa, New Zealand and Australia from December 2006 to July 2010.”

• “Patients were centrally randomly assigned 1:1 using an interactive voice response system to adalimumab (40 mg every other week) plus methotrexate (titrated to 20 mg/week by week 8) or PBO+MTX for 26 weeks.”

Reasoning: The trial was conducted at multiple centers across different continents, suggesting a diverse range of recruitment sites. The use of an interactive voice response system for randomization indicates a systematic and standardized approach to participant recruitment. Score: [4]

3. Setting: Where is the trial being done?

Quotes:

• “OPTIMA (NCT00420927) was conducted at 161 centres in North and South America, Europe, Africa, New Zealand and Australia from December 2006 to July 2010.”

• “A central institutional review board or independent ethics committee approved the study at each site; the study was conducted in accordance with the International Conference on Harmonisation good clinical practices and the Declaration of Helsinki.”

Reasoning: The trial was conducted at multiple centers across different continents, indicating a wide range of settings. The approval of the study by central institutional review boards or independent ethics committees suggests that the trial was conducted in accordance with ethical guidelines. Score: [4]

4. Organisation: What expertise and resources are needed to deliver the intervention?

Quotes:

• “Patients with early active RA were randomly assigned to receive adalimumab plus methotrexate (ADA+MTX) or placebo plus methotrexate (PBO+MTX) for 26 weeks.”

• “Treatment modifications were to be made in a subsequent study period based on the achievement of DAS28(CRP) <3.2 at weeks 22 and 26.”

Reasoning: The intervention in this trial involved the use of adalimumab plus methotrexate or placebo plus methotrexate. The trial design allowed for treatment modifications based on the achievement of specific disease activity targets. This suggests that the intervention could be delivered within the usual care setting and did not require additional expertise or resources beyond what is typically available for the treatment of early active RA. Score: [5]

5. Flexibility (delivery): How should the intervention be delivered?

Quotes:

• “Treatment modifications were to be made in a subsequent study period based on the achievement of DAS28(CRP) <3.2 at weeks 22 and 26.”

• “Post-hoc analyses compared patients achieving stable remission using DAS28 and 2010 ACR/EULAR criteria with those achieving LDA but not remission.”

Reasoning: The trial allowed for treatment modifications based on the achievement of specific disease activity targets, indicating flexibility in the delivery of the intervention. Post-hoc analyses also compared different levels of disease activity response, suggesting that the intervention could be delivered in a flexible manner based on individual patient response. Score: [4]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

Quotes:

• “Among patients completing 6 months, 44% (207/466) ADA+MTX versus 24% (112/460) PBO+MTX patients achieved stable LDA at weeks 22 and 26 (p<0.001).”

• “Patients achieving ACR/EULAR remission, particularly in the PBO+MTX group, had some advantage in radiographic outcomes compared with patients who only achieved LDA (but not remission).”

Reasoning: The trial reported the percentage of patients who achieved stable low disease activity (LDA) at specific time points, indicating a measure of adherence to the intervention. The comparison of outcomes between patients who achieved different levels of disease activity response also suggests an assessment of adherence to the intervention. Score: [4]

7. Follow-up: How closely are participants followed up?

Quotes:

• “Clinical assessments during period 1 were performed at baseline and weeks 2, 4, 8, 12, 22 and 26.”

• “Radiographs of the hands/wrists and feet were obtained at baseline and week 26, and mean changes in the mTSS, joint erosion and joint space narrowing were used to evaluate radiographic progression; no radiographic progression was defined as ΔmTSS of 0.5 or less.”

Reasoning: Participants in the trial were followed up at regular intervals, with clinical assessments and radiographs performed at specific time points. The use of radiographs to evaluate radiographic progression indicates a thorough follow-up process. Score: [4]

8. Primary outcome: How relevant is it to participants?

Quotes:

• “The primary endpoint of the study was a composite of DAS28(CRP) less than 3.2 at week 78 and no radiographic progression from baseline to week 78 in the van der Heijde modified total Sharp score (ΔmTSS ≤0.5).”

• “In this report, the percentage of patients achieving a stable LDA target of DAS28(CRP) less than 3.2 at weeks 22 and 26 was compared between treatment groups.”

Reasoning: The primary outcome of the trial was a composite measure that included disease activity and radiographic progression, which are relevant outcomes for participants with early rheumatoid arthritis. The use of a composite measure that includes both clinical and radiographic outcomes suggests a comprehensive assessment of treatment efficacy. Score: [4]

9. Primary analysis: To what extent are all data included?

Quotes:

• “The intent-to-treat population included all randomly assigned patients who received at least one dose of study drug.”

• “Last observation carried forward analyses were used for continuous clinical and functional outcomes.”

Reasoning: The trial included an intent-to-treat population, which indicates that all randomly assigned patients who received at least one dose of the study drug were included in the analysis. The use of last observation carried forward analyses for continuous outcomes suggests that all available data were included in the analysis. Score: [5]

Homeopathic ear drops as an adjunct to standard therapy in children with acute otitis media (2011), PragEpi

1. Eligibility criteria: Who is selected to participate in the trial?

Quotes:

• “Children with AOM were enrolled in the study at the time of diagnosis and randomized to receive either standard therapy alone or standard therapy plus a homeopathic ear drop solution that was to be used on as needed basis for up to 5 days.”

• “Children with a chronic medical condition, those who had received antibiotics within the previous 2 days, had a diagnosis of AOM during the preceding 30 days, or who had a perforated tympanic membrane were excluded.”

Reasoning: The eligibility criteria for this trial seem to be pragmatic. They include children with AOM who are diagnosed at the time of enrollment and exclude those with certain medical conditions or recent antibiotic use. This reflects the usual care setting for AOM treatment.

Score: [4]

2. Recruitment: How are participants recruited into the trial?

Quotes:

• “When a diagnosis of AOM in an age eligible child was made during the study period, the examining provider determined the appropriate treatment for the patient.”

• “In addition, the provider briefly described the study to the parent at the visit.”

Reasoning: The recruitment strategy for this trial appears to be pragmatic. Participants are recruited during routine clinical visits when a diagnosis of AOM is made. The trial is described to the parents by the healthcare provider, which reflects the usual care setting.

Score: [4]

3. Setting: Where is the trial being done?

Quotes:

• “The study was conducted at the University of Washington Medical Center Pediatric Care Center, a comprehensive primary care clinic that is a major training site for pediatric residents.”

Reasoning: The trial was conducted in a primary care clinic, which is a usual care setting for the treatment of AOM. This reflects a pragmatic approach to the choice of setting.

Score: [4]

4. Organisation: What expertise and resources are needed to deliver the intervention?

Quotes:

• “The homeopathic ear drops, Hylands Earache Drops, used in the study are commercially available in the United States.”

• “Parents of children randomized to the ear drops were instructed to administer 3-4 drops up to 3 times/day as needed for relief of AOM symptoms for a maximum of 5 days.”

Reasoning: The intervention in this trial involves the use of commercially available homeopathic ear drops, which suggests that it can be easily implemented in usual care settings. The instructions for administration are straightforward and do not require additional expertise or resources.

Score: [5]

5. Flexibility (delivery): How should the intervention be delivered?

Quotes:

• “Parents of all study patients were asked to complete a symptom severity diary twice daily for the first 5 days after enrollment.”

• “Regardless of group assignment, parents of all study patients were asked to complete a symptom severity diary twice daily for the first 5 days after enrollment.”

Reasoning: The delivery of the intervention in this trial appears to be flexible. Parents are asked to complete a symptom severity diary twice daily, which reflects the usual care practice of monitoring symptoms in children with AOM.

Score: [5]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

Quotes:

• “Parents of all study patients were asked to complete a symptom severity diary twice daily for the first 5 days after enrollment.”

• “Data on the occurrence of ‘other symptoms’ were considered to be the main assessment of adverse events related to treatment.”

Reasoning: The trial does not have specific measures in place to ensure adherence to the intervention. Participants are asked to complete a symptom severity diary, but there is no mention of consequences for non-adherence. This suggests a pragmatic approach to adherence.

Score: [4]

7. Follow-up: How closely are participants followed up?

Quotes:

• “Parents of all study patients were asked to complete a symptom severity diary twice daily for the first 5 days after enrollment.”

• “Approximately 12-15 days after enrollment parents of study children were contacted by telephone for a follow-up assessment.”

Reasoning: Participants in this trial are followed up closely through symptom severity diaries completed twice daily for the first 5 days after enrollment. There is also a follow-up assessment conducted by telephone approximately 12-15 days after enrollment. This reflects a pragmatic approach to follow-up.

Score: [4]

8. Primary outcome: How relevant is it to participants?

Quotes:

• “Primary study outcomes included the ETG-5 scores at each assessment after enrollment and occurrence of adverse events.”

• “The ETG-5 was developed to quantify the severity of specific symptoms in children with AOM.”

Reasoning: The primary outcome measure in this trial, the ETG-5 scores, is directly relevant to participants as it assesses the severity of specific symptoms in children with AOM. This reflects a pragmatic approach to selecting a primary outcome.

Score: [5]

9. Primary analysis: To what extent are all data included?

Quotes:

• “Primary study outcomes included the ETG-5 scores at each assessment after enrollment and occurrence of adverse events.”

• “Mann-Whitney tests were used to assess the statistical significance of differences in ETG-5 scores between children randomized to ear drops and those receiving standard therapy alone.”

Reasoning: The primary analysis in this trial includes all data on the primary outcomes, the ETG-5 scores, and adverse events. The statistical analysis used, Mann-Whitney tests, is appropriate for analyzing non-parametric data. This reflects a pragmatic approach to the primary analysis.

Score: [5]

Resection of Cavity Shave Margins in Stage 0-III Breast Cancer Patients Undergoing Breast Conserving Surgery: A Prospective Multicenter Randomized Controlled Trial (2019), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

• Quote 1: “This trial enrolled 400 female Stage 0–III breast cancer patients aged 18 years or older undergoing partial mastectomy.”

• Quote 2: “Exclusion criteria included excisional biopsy for diagnosis, prior attempt at partial mastectomy, bilateral breast cancer, and plan for intraoperative radiation therapy.”

• Quote 3: “Patients were screened for eligibility at each site, and written informed consent was obtained.”

Score: [4] The full text provides clear information about the eligibility criteria for the trial, including the specific patient population, exclusion criteria, and the process of obtaining informed consent.

2. Recruitment: How are participants recruited into the trial?

• Quote 1: “In this multicenter randomized controlled trial occurring in 9 centers across the United States, stage 0–III breast cancer patients undergoing PM were randomly assigned to either have resection of CSM (“shave” group) or not (“no shave” group).”

• Quote 2: “Randomization occurred intraoperatively, after the surgeon had completed their standard PM.”

Score: [4] The full text indicates that participants were recruited from multiple centers across the United States and were randomly assigned to the intervention or control group. It also mentions that randomization occurred intraoperatively, which suggests that participants were recruited during their surgical procedures.

3. Setting: Where is the trial being done?

• Quote 1: “Nine centers from across the United States participated in this prospective randomized controlled trial.”

• Quote 2: “This study was approved by the Yale University Human Investigations Committee, and each of the participating site’s Institutional Review Board.”

Score: [4] The full text clearly states that the trial was conducted in nine centers across the United States. It also mentions the approval of the study by the relevant ethics committees.

4. Organisation: What expertise and resources are needed to deliver the intervention?

• Quote 1: “Surgeons were instructed to perform partial mastectomy according to their usual practice, including excision of selective margins based on intraoperative imaging and/or their own gross evaluation.”

• Quote 2: “Once the surgeon completed the partial mastectomy, a phone call was placed to the coordinating center at Yale and the patient’s randomization group was revealed to the surgical team.”

Score: [4] The full text indicates that the intervention (resection of CSM) was delivered by surgeons according to their usual practice, including the use of selective margins. It also mentions the involvement of a coordinating center at Yale, suggesting that the trial had the necessary expertise and resources to deliver the intervention.

5. Flexibility (delivery): How should the intervention be delivered?

• Quote 1: “Surgeons were instructed to take CSM encompassing the entire partial mastectomy cavity. Superior, medial, inferior, and lateral margins were mandated; anterior and posterior margins could be omitted if resection had extended to the skin or pectoralis fascia, respectively.”

• Quote 2: “Partial mastectomy specimens were processed per local policies, serially sectioned for gross evaluation, with representative sections submitted for histologic evaluation.”

Score: [3] The full text provides some information about the delivery of the intervention, including the specific instructions given to surgeons regarding the resection of CSM. However, it does not provide details about the flexibility allowed in the delivery of the intervention or whether any deviations from the protocol were permitted.

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• Quote 1: “Re-excision was left to the discretion of the surgeon.”

• Quote 2: “Of the 91 patients who had a positive final margin, 61 (67.0%) underwent a re-excision. Re-excision was left to the discretion of the surgeon.”

Score: [4] The full text mentions that re-excision was left to the discretion of the surgeon, indicating that there was flexibility in how participants adhered to the intervention. However, it does not provide information about any specific measures or strategies in place to ensure adherence.

7. Follow-up: How closely are participants followed up?

• Quote 1: “Median follow-up time was 13 months (range, 4–31).”

• Quote 2: “Cosmetic outcome and quality of life (each measured at 1- and 5-yrs), and 5-year local recurrence rate will be reported in due course.”

Score: [3] The full text provides information about the duration of follow-up (median of 13 months) and mentions that cosmetic outcome and quality of life will be measured at 1- and 5-year intervals. However, it does not provide details about the frequency or intensity of follow-up visits or any specific measures in place to ensure participant compliance with follow-up.

8. Primary outcome: How relevant is it to participants?

• Quote 1: “Primary outcome measures were positive margin and re-excision rates.”

• Quote 2: “Resection of CSM significantly reduces positive margin and re-excision rates in patients undergoing PM.”

Score: [5] The full text clearly states that the primary outcome measures (positive margin and re-excision rates) are directly relevant to the participants. It also provides evidence that resection of CSM has a significant impact on these outcomes.

9. Primary analysis: To what extent are all data included?

• Quote 1: “Group comparisons were made using Fisher exact or chi square tests for categorical variables and Mann–Whitney U tests for continuous variables.”

• Quote 2: “IBM SPSS Statistics version 24.0 was used for statistical analysis.”

Score: [4] The full text mentions the statistical methods used for the primary analysis and the software used for the analysis. However, it does not provide information about whether all available data were included in the analysis or any specific measures taken to address missing data.

Improving adherence to cardiovascular disease medications with information technology (2014), PragEpi

1. Eligibility criteria: Who is selected to participate in the trial?

• “We enrolled participants who were 40 years or older, had diabetes mellitus or atherosclerotic cardiovascular disease, and were suboptimally adherent.” [1]

• “Both interventions significantly increased adherence to statins and angiotensin-converting enzyme inhibitors/angiotensin receptor blockers (ACEIs/ARBs) compared with UC (1.6 to 3.7 percent-age points).” [2]

• “Adherence to ACEIs/ARBs was also significantly higher for IVR+ relative to IVR participants.” [2]

The full text provides information on the eligibility criteria for the trial. Participants were required to be 40 years or older, have diabetes mellitus or atherosclerotic cardiovascular disease, and be suboptimally adherent to their medications. The trial included interventions to improve adherence to statins and ACEIs/ARBs. The interventions were found to significantly increase adherence compared to usual care. Based on this information, the trial can be considered pragmatic as it included a broad range of participants who would likely be candidates for the intervention in usual care. Score: [5]

2. Recruitment: How are participants recruited into the trial?

• “UC participants had access to the full range of usual services, including each region’s normal education and care management outreach efforts to encourage statin and ACEI/ARB use.”

• “IVR participants received automated phone calls when they were due or overdue for a refill.”

• “IVR+ participants received a personalized reminder letter if they were 60 to 89 days overdue and a live outreach call if they were ≥90 days overdue, as well as EMR-based feedback to their primary care provider.”

The full text describes the recruitment strategies used in the trial. UC participants received usual care services, including education and care management outreach efforts. IVR participants received automated phone calls for medication refills, while IVR+ participants received additional personalized reminders, live outreach calls, and feedback to their primary care providers. These recruitment strategies are pragmatic as they are similar to what would be done in usual care. Score: [5]

3. Setting: Where is the trial being done?

• “The study took place in 3 large health maintenance organizations.”

• “Participants were members of one of 3 regions of the Kaiser Permanente (KP) health plan—Northwest (KPNW), Hawaii (KPH), and Georgia (KPG)—which collectively serve about 944,000 individuals.”

The full text provides information on the setting of the trial. It was conducted in three large health maintenance organizations within the Kaiser Permanente health plan. The trial was conducted in real-world settings and included a large number of participants. Based on this information, the trial can be considered pragmatic. Score: [5]

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “UC participants had access to the full range of usual services, including each region’s normal education and care management outreach efforts to encourage statin and ACEI/ARB use.”

• “IVR participants received automated phone calls when they were due or overdue for a refill.”

• “IVR+ participants received a personalized reminder letter if they were 60 to 89 days overdue and a live outreach call if they were ≥90 days overdue, as well as EMR-based feedback to their primary care provider.”

The full text does not provide specific information on the expertise and resources needed to deliver the intervention. However, it mentions that UC participants had access to usual services, including education and care management outreach efforts. IVR participants received automated phone calls, while IVR+ participants received additional personalized reminders, live outreach calls, and feedback to their primary care providers. Based on this information, it can be inferred that the intervention was designed to be integrated into usual care and did not require additional expertise or resources. Score: [4]

5. Flexibility (delivery): How should the intervention be delivered?

• “IVR participants received automated phone calls when they were due or overdue for a refill.”

• “IVR+ participants received a personalized reminder letter if they were 60 to 89 days overdue and a live outreach call if they were ≥90 days overdue, as well as EMR-based feedback to their primary care provider.”

The full text provides information on how the intervention was delivered. IVR participants received automated phone calls for medication refills, while IVR+ participants received personalized reminder letters and live outreach calls. The delivery of the intervention was not rigidly prescriptive and allowed for flexibility in how participants engaged with it. Based on this information, the trial can be considered pragmatic. Score: [5]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• “IVR participants received automated phone calls when they were due or overdue for a refill.”

• “IVR+ participants received a personalized reminder letter if they were 60 to 89 days overdue and a live outreach call if they were ≥90 days overdue, as well as EMR-based feedback to their primary care provider.”

The full text provides information on the measures in place to ensure participants adhere to the intervention. IVR participants received automated phone calls for medication refills, while IVR+ participants received personalized reminder letters and live outreach calls. These measures were designed to improve adherence to medication refills. Based on this information, the trial can be considered pragmatic. Score: [5]

7. Follow-up: How closely are participants followed up?

• “Mean duration of follow-up was 9.6 months and did not vary by treatment arm.”

• “During the 1-year follow-up period, 427 study participants (2.0%) died; 0.3% of participants were hospitalized for conditions potentially related to ACEI/ARB use; and 0.02% were hospitalized for conditions potentially related to statin use.”

The full text provides information on the follow-up of participants in the trial. The mean duration of follow-up was 9.6 months and did not vary by treatment arm. The trial also reported on participant safety outcomes during the follow-up period. Based on this information, the trial can be considered pragmatic. Score: [5]

8. Primary outcome: How relevant is it to participants?

• “The primary outcome was medication adherence; blood pressure and lipid levels were secondary outcomes.”

• “Both interventions significantly increased adherence to statins and angiotensin-converting enzyme inhibitors/angiotensin receptor blockers (ACEIs/ARBs) compared with UC (1.6 to 3.7 percent-age points).”

• “Among statin users, IVR+ participants had significantly lower low-density lipoprotein (LDL) levels at follow-up compared with UC (Δ = −1.5; 95% CI, −2.7 to −0.2 mg/dL); this effect was seen mainly in those with baseline LDL levels >100 mg/dL (Δ = −3.6; 95% CI, −5.9 to −1.3 mg/dL).”

The full text provides information on the primary outcome of the trial, which was medication adherence. The trial also measured secondary outcomes such as blood pressure and lipid levels. These outcomes are relevant to participants as they are directly related to their cardiovascular health. Based on this information, the trial can be considered pragmatic. Score: [5]

9. Primary analysis: To what extent are all data included?

• “We used an intention-to-treat analysis to compare primary and secondary outcomes between intervention and UC participants.”

• “All adherence analyses were conducted separately for users of statins and users of ACEIs/ARBs.”

• “We used logistic regression for analyses of BP control and LDL-cholesterol control.”

The full text provides information on the primary analysis conducted in the trial. An intention-to-treat analysis was used to compare primary and secondary outcomes between intervention and UC participants. The analysis included all available data and used appropriate statistical methods. Based on this information, the trial can be considered pragmatic. Score: [5]

Effectiveness of Fibrin Sealant Application on the Development of Staple Line Complications After Sleeve Gastrectomy: A Prospective Randomized Trial (2018), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

• “During a consultation before bariatric surgery, patients meeting the study’s inclusion criteria were invited to participate in the study and were also provided with an information sheet on the study objectives and procedures.”

• “The inclusion criteria were a BMI between 35 and 60 kg/m2, age between 18 and 60 years, and primary SG. The exclusion criteria were history of bariatric or gastric surgery, a surgical procedure other than primary SG, a BMI ≥60 kg/m2, age under 18, allergy or contraindication to TISSEEL, legal guardianship, refusal to participate in the study, pregnancy or breastfeeding, and an American Society of Anesthesiologists score of IV.”

• “Patients with a positive intraoperative methylene blue test and/or abdominal drainage and/or intraoperative bleeding requiring staple line suturing were not excluded.”

The full text provides information on the eligibility criteria for the trial. It states that patients meeting the inclusion criteria (BMI, age, and primary SG) were invited to participate, while those who met the exclusion criteria (history of bariatric or gastric surgery, BMI ≥60 kg/m2, etc.) were not included. Additionally, patients with specific intraoperative findings (positive methylene blue test, abdominal drainage, or bleeding requiring suturing) were not excluded. Based on this information, it can be inferred that the trial had relatively inclusive eligibility criteria, which suggests a more pragmatic approach. Score: [4]

2. Recruitment: How are participants recruited into the trial?

• “During a consultation before bariatric surgery, patients meeting the study’s inclusion criteria were invited to participate in the study and were also provided with an information sheet on the study objectives and procedures.”

• “Randomization was performed the day before surgery. The sequence of study treatment allocations (FS or not) was generated by a data manager using a computer.”

The full text mentions that patients meeting the inclusion criteria were invited to participate during a consultation before surgery. Randomization was performed the day before surgery using a computer-generated sequence. This suggests that participants were recruited from the pool of patients scheduled for bariatric surgery, which is a pragmatic approach. Score: [5]

3. Setting: Where is the trial being done?

• “This was a prospective, intention-to-treat, randomized, single-blind, 2-center superiority study of a group of patients (n = 586) undergoing primary SG between February 2014 and June 2017.”

The full text states that the trial was conducted at two centers. However, it does not provide any further information about the characteristics of the centers or how they relate to the setting where the results will be applied. Without additional information, it is difficult to assess the match between the trial setting and the intended setting of application. Score: [NA]

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “The 4-trocar SG technique has been described in detail by Dhahri et al.20 A 34 Fr bougie guided the stapling, and stapling was started 6 cm proximal to the pylorus using a Endo Gia Ultra Universal XL Stapler 60 with Tri-staple purple reloads (Covidien France SAS, Elancourt, France) or an Echelon Flex Endopath 60-mm stapler with green reloads (Ethicon Endo-Surgery, Issy-les-Moulineaux, France). No buttress material or GSL over sewing was used.”

• “Fibrin sealant (Tisseel, Baxter International Inc., Westlake Village, CA) was applied after the intraoperative methylene blue test had been performed.”

The full text describes the surgical technique used for sleeve gastrectomy, including the specific staplers and sealant used. It does not mention any additional expertise or resources needed beyond the standard surgical equipment and materials. This suggests a more pragmatic approach to the organization of care. Score: [5]

5. Flexibility (delivery): How should the intervention be delivered?

• “The FS was sprayed on the GSL using two 2 mL syringes per patient, a DUPLOSPRAY MIS gas-assisted endoscopic applicator (CO2 pressure: between 51 and 100 pounds per square inch) (Baxter International Inc.) and a 20 cm trocar cannula.”

The full text provides details on how the fibrin sealant intervention was delivered, including the specific equipment used. It does not mention any restrictions or specifications for the delivery of the intervention beyond these details. This suggests a more pragmatic approach with flexibility in the delivery of the intervention. Score: [5]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• No relevant quotes found.

The full text does not provide any information on measures to ensure participant adherence to the intervention. Without this information, it is difficult to assess the level of flexibility in adherence. Score: [NA]

7. Follow-up: How closely are participants followed up?

• “SLCs were evaluated over the first 30 postoperative days (PODs).”

• “Loss to follow-up was defined as the absence of a consultation on POD 30.”

The full text states that participants were followed up for 30 postoperative days, and loss to follow-up was defined as the absence of a consultation on POD 30. This suggests a relatively short follow-up period, which may limit the ability to capture long-term outcomes. However, without additional information on the frequency and nature of follow-up visits, it is difficult to assess the level of follow-up intensity. Score: [3]

8. Primary outcome: How relevant is it to participants?

• “The trial’s primary endpoint was a composite of the SLCs recorded over the first 30 PODs.”

The full text states that the primary endpoint of the trial was a composite of staple line complications recorded over the first 30 postoperative days. This outcome is directly relevant to the participants as it captures the occurrence of complications related to the surgical procedure. Score: [5]

9. Primary analysis: To what extent are all data included?

• “An intention-to-treat analysis was performed.”

The full text mentions that an intention-to-treat analysis was performed, which suggests that all available data were included in the analysis. This approach is consistent with a pragmatic design. Score: [5]

Total extraperitoneal inguinal hernia repair compared with Lichtenstein (the LEVEL-Trial): a randomized controlled trial (2010), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

• “Adults presenting with primary or recurrent inguinal hernias, uni- or bilateral, with an indication for elective correction, were eligible for randomization.” [1]

• “Exclusion criteria were scrotal hernia, pregnancy, and communicative or cognitive limitation that prevented informed consent.” [1]

• “A medical history of prostatectomy, pfannenstiel incision, preperitoneal procedure, or abdominal bladder operation were other reasons for exclusion.” [1]

The full text provides clear information on the eligibility criteria for the trial. The trial included adults with primary or recurrent inguinal hernias, uni- or bilateral, who were candidates for elective correction. Exclusion criteria were also clearly defined. Based on this information, the trial appears to have a pragmatic approach to eligibility criteria. Score: [4]

2. Recruitment: How are participants recruited into the trial?

• “Randomization for TEP or Lichtenstein was carried out by telephone or fax at the outpatient clinic, by the central study coordinator, according to a balanced and stratified computer-generated list.” [1]

The full text mentions that randomization for the trial was carried out by the central study coordinator using a computer-generated list. However, there is no information provided on how participants were initially identified or recruited into the trial. Without this information, it is difficult to assess the recruitment domain. Score: [NA]

3. Setting: Where is the trial being done?

• “Six hospitals, including 1 university hospital in the Netherlands participated in the “LEVEL”-trial.” [1]

The full text states that the trial was conducted in six hospitals, including one university hospital in the Netherlands. While the specific characteristics of the hospitals and their match to the setting where the results will be applied are not provided, the inclusion of multiple hospitals suggests a more pragmatic approach to the setting. Score: [3]

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “Surgeons and residents were experienced with both procedures or were supervised by an experienced surgeon.” [1]

The full text mentions that surgeons and residents involved in the trial were experienced with both procedures or were supervised by an experienced surgeon. This suggests that the trial aimed to use existing expertise and resources in delivering the interventions, which aligns with a pragmatic approach. Score: [4]

5. Flexibility (delivery): How should the intervention be delivered?

• “Instructions were given to all patients, to resume normal daily activities and return to work as soon as possible, irrespective of the operative method.” [1]

The full text mentions that patients were instructed to resume normal daily activities and return to work as soon as possible, regardless of the operative method. This suggests a flexible approach to the delivery of the intervention, allowing for individualized implementation based on usual care practices. Score: [4]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• No relevant quotes found.

The full text does not provide any information on measures in place to ensure participant adherence to the intervention. Without this information, it is difficult to assess the flexibility (adherence) domain. Score: [NA]

7. Follow-up: How closely are participants followed up?

• “Follow-up after 6 weeks was completed in 89% (568) and 1 year in 77% (495).” [1]

The full text mentions that follow-up was completed at 6 weeks and 1 year, with completion rates of 89% and 77% respectively. While the follow-up rates are not ideal, they are still relatively high and suggest a pragmatic approach to follow-up. Score: [3]

8. Primary outcome: How relevant is it to participants?

• “Primary endpoints were postoperative pain, length of hospital stay, period until complete recovery, and quality of life.” [1]

The full text clearly states the primary endpoints of the trial, which include outcomes that are relevant to participants such as postoperative pain, length of hospital stay, period until complete recovery, and quality of life. These outcomes align with a pragmatic approach. Score: [4]

9. Primary analysis: To what extent are all data included?

• “Primary analysis was performed on an intention to treat basis.” [1]

The full text mentions that the primary analysis was performed on an intention-to-treat basis, which means that all data were included regardless of adherence or other factors. This approach aligns with a pragmatic design. Score: [5]

Head-to-head comparison of subcutaneous abatacept versus adalimumab for rheumatoid arthritis: two-year efficacy and safety findings from AMPLE trial (2014), PragEpi

1. Eligibility criteria: Who is selected to participate in the trial?

• “Biologic-naive patients with active RA and an inadequate response to MTX were randomised to 125 mg abatacept weekly or 40 mg adalimumab bi-weekly, both with a stable dose of MTX.”

Score: [5]

The full text clearly states that the trial included patients with active RA and an inadequate response to MTX. The eligibility criteria are well-defined and align with the usual care for patients with this condition. Therefore, the trial scores a 5 on the PRECIS-2 scale for this domain.

2. Recruitment: How are participants recruited into the trial?

• “Patients were equally assigned to receive 125 mg abatacept (Orencia; Bristol-Myers Squibb), administered subcutaneous weekly (without an intravenous loading dose), or 40 mg adalimumab (Humira; Abbott Laboratories), administered subcutaneous every other week, both in combination with a stable dose of MTX (≥15 and ≤25 mg/week or ≥7.5 mg/week if documented intolerance to higher doses).”

Score: [5]

The full text states that patients were equally assigned to receive either abatacept or adalimumab, both in combination with a stable dose of MTX. This approach is pragmatic as it reflects the usual care for patients with RA who have an inadequate response to MTX. Therefore, the trial scores a 5 on the PRECIS-2 scale for this domain.

3. Setting: Where is the trial being done?

• “AMPLE is a 2-year, phase IIIB, multinational, prospective, randomised study comparing subcutaneous abatacept and adalimumab on stable background MTX in patients naive to bDMARDs and is the only one of these comparative trials to date to also include radiographic assessment.”

Score: [4]

The full text mentions that the trial is multinational, which suggests that it was conducted in multiple settings. However, it does not provide specific information about the settings where the trial was conducted. Therefore, the score for this domain is a 4.

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “Patients were equally assigned to receive 125 mg abatacept (Orencia; Bristol-Myers Squibb), administered subcutaneous weekly (without an intravenous loading dose), or 40 mg adalimumab (Humira; Abbott Laboratories), administered subcutaneous every other week, both in combination with a stable dose of MTX.”

Score: [5]

The full text states that patients were assigned to receive either abatacept or adalimumab, both in combination with a stable dose of MTX. This approach is pragmatic as it reflects the usual care for patients with RA who have an inadequate response to MTX. Therefore, the trial scores a 5 on the PRECIS-2 scale for this domain.

5. Flexibility (delivery): How should the intervention be delivered?

• “Patients were equally assigned to receive 125 mg abatacept (Orencia; Bristol-Myers Squibb), administered subcutaneous weekly (without an intravenous loading dose), or 40 mg adalimumab (Humira; Abbott Laboratories), administered subcutaneous every other week, both in combination with a stable dose of MTX.”

Score: [5]

The full text does not provide specific information about how the intervention was delivered. However, it does mention that patients were assigned to receive either abatacept or adalimumab, both in combination with a stable dose of MTX. This suggests that the intervention was delivered in a similar way to usual care, which is a pragmatic approach. Therefore, the trial scores a 5 on the PRECIS-2 scale for this domain.

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• No relevant quotes were found in the full text.

Score: [NA]

The full text does not provide any information about the measures in place to ensure participants adhere to the intervention. Therefore, it is not possible to assess the score for this domain.

7. Follow-up: How closely are participants followed up?

• “Efficacy and safety outcomes were evaluated at days 1, 15, 29 and every 4 weeks thereafter during year 1, and every 3 months during year 2.”

Score: [NA]

The full text does not provide any information about how closely participants were followed up. Therefore, it is not possible to assess the score for this domain.

8. Primary outcome: How relevant is it to participants?

• “The primary endpoint was at 1 year but the blinded study continued for 2 years to provide controlled, comparative assessment of long term safety, efficacy and radiographic outcomes.”

Score: [NA]

The full text does not provide any information about the relevance of the primary outcome to participants. Therefore, it is not possible to assess the score for this domain.

9. Primary analysis: To what extent are all data included?

• “All efficacy analyses were assessed using the intent-to-treat (ITT) populations; all safety analyses were assessed using the as-treated population.”

Score: [NA]

The full text does not provide any information about the extent to which all data were included in the primary analysis. Therefore, it is not possible to assess the score for this domain.

Effects of treatment on morbidity in hypertension. Results in patients with diastolic blood pressures averaging 115 through 129 mm Hg (1967), PragEpi

1. Eligibility criteria: Who is selected to participate in the trial?

Quotes: “Male patients with diastolic blood pressures averaging 115 mm Hg or above represent a high-risk group in which antihypertensive therapy exerts a significant beneficial effect.” “Also excluded were patients with surgically curable hypertension, uremia, and concomitant fatal diseases such as carcinoma.” “Patients with hemorrhages, exudates, or papilledema in the optic fundi, history of cerebral or subarachnoid hemorrhage, dissecting aneurysm, or congestive heart failure resistant to digitalis and mercurial diuretics were excluded.”

Reasoning: The eligibility criteria for this trial were clearly defined and focused on a specific population of male hypertensive patients with diastolic blood pressures averaging 115 mm Hg or above. The exclusion criteria were also well-defined and aimed to exclude patients with certain conditions that could confound the results. Overall, the trial had a clear focus on a specific patient population, which suggests a more pragmatic approach.

Score: [4]

2. Recruitment: How are participants recruited into the trial?

Quotes: “Following discharge from the hospital, the patients entered a prerandomization trial period of two to four months’ duration.” “Patients were required to return all bottles of medication, at which time the tablets were counted to assess the patient’s reliability.” “Nearly one half of the patients accepted into the trial period were excluded prior to randomization because of failure to pass the above tests of reliability.”

Reasoning: The recruitment process involved a prerandomization trial period during which patients were assessed for their reliability and adherence to the protocol. Patients were required to return medication bottles and undergo tablet counts to assess their reliability. The trial had a rigorous screening process to ensure that only reliable and adherent patients were included. This suggests a more explanatory approach to recruitment.

Score: [2]

3. Setting: Where is the trial being done?

Quotes: “The study was terminated in the 115 mm Hg and above group at an earlier date than expected when it became apparent that the risk rate increased sharply at these levels of diastolic blood pressure and that the clinical course of such patients appeared to be favorably influenced by antihypertensive drug treatment.” “The patients visited the clinic at monthly intervals for the first two months following randomization and at bimonthly intervals henceforth.”

Reasoning: The trial was conducted in multiple clinics, as patients visited the clinic at regular intervals for follow-up. The decision to terminate the study in the high-risk group with diastolic blood pressures of 115 mm Hg and above suggests that the trial was conducted in a setting where patients with this level of hypertension would typically receive treatment. This indicates a more pragmatic approach to the choice of setting.

Score: [4]

4. Organisation: What expertise and resources are needed to deliver the intervention?

Quotes: “Treatment was begun with either tablet A, one twice daily, plus the 25-mg tablet B, one tablet three times daily, or else with the placebos of both of these tablets.” “Dosages were reduced in the remaining patients because of low blood pressure levels with the standard regimen or because of side effects such as severe headache or weakness.”

Reasoning: The intervention involved the administration of specific tablets (tablet A and tablet B) at specified dosages. The dosages could be adjusted based on the patient’s blood pressure levels or side effects. The intervention did not require additional resources or expertise beyond the administration of the tablets. This suggests a more pragmatic approach to the organization of care.

Score: [5]

5. Flexibility (delivery): How should the intervention be delivered?

Quotes: “Treatment was begun with either tablet A, one twice daily, plus the 25-mg tablet B, one tablet three times daily, or else with the placebos of both of these tablets.” “Thus, the regular maintenance daily dose to patients randomized to the active regimen was 100 mg of hydrochlorothiazide, 0.2 mg of reserpine, and 150 mg of hydralazine hydrochloride.”

Reasoning: The intervention was delivered through the administration of specific tablets at specified dosages. The protocol did not provide rigid instructions on how the intervention should be delivered beyond the dosage instructions. This suggests a more pragmatic approach to the delivery of the intervention.

Score: [5]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

Quotes: “Patients were required to return all bottles of medication, at which time the tablets were counted to assess the patient’s reliability.” “To qualify for admission to the study, the patients were required to have no ‘violations’ on two successive clinic visits.”

Reasoning: The trial had measures in place to assess and ensure patient adherence to the intervention. Patients were required to return medication bottles for tablet counts, and their reliability was assessed based on their adherence to the protocol. This suggests a more explanatory approach to adherence.

Score: [2]

7. Follow-up: How closely are participants followed up?

Quotes: “The patients visited the clinic at monthly intervals for the first two months following randomization and at bimonthly intervals henceforth.” “Tablet counts were made at all clinic visits and fluorescence tests of the urine were made at alternate visits.”

Reasoning: Participants were followed up regularly at clinic visits, with more frequent visits in the initial months and less frequent visits thereafter. Tablet counts and urine tests were conducted at these visits to monitor adherence. The follow-up schedule and assessments were consistent with usual care for hypertensive patients. This suggests a more pragmatic approach to follow-up.

Score: [4]

8. Primary outcome: How relevant is it to participants?

Quotes: “An investigation of this type was initiated by the Veterans Administration Cooperative Study Group on Antihypertensive Agents in 1963.” “The present report is concerned with the results obtained in the patients without signs of accelerated hypertension at admission whose diastolic blood pressures prior to treatment averaged 115 through 129 mm Hg.”

Reasoning: The primary outcome of the trial was the occurrence of severe complications and mortality in patients with diastolic blood pressures averaging 115 mm Hg or above. This outcome is directly relevant to the participants as it measures the impact of the intervention on their health and well-being. This suggests a more pragmatic approach to the choice of primary outcome.

Score: [5]

9. Primary analysis: To what extent are all data included?

Quotes: “Changes in systolic (left) and diastolic blood pressure (right) after four months of treatment in 57 patients given placebos (above) and 68 patients treated with hydrochlorothiazide plus reserpine plus hydralazine (below).” “As shown in Table 4 assessable morbid events occurred in 27 placebo-treated versus two actively treated patients.”

Reasoning: The primary analysis included all available data on changes in blood pressure and the occurrence of morbid events. The results were reported for both the placebo-treated and actively treated groups. This suggests an intention-to-treat analysis approach, which is consistent with a more pragmatic design.

Score: [5]

Metal-carbon fiber composite femoral stems in hip replacements: a randomized controlled parallel-group study with mean ten-year follow-up (2014), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

• “Subjects between fifty and seventy years of age who presented for primary total hip arthroplasty for noninflammatory arthritis at a single regional orthopaedic center were eligible for enrollment into the study.” [1]

• “Exclusion criteria included inflammatory arthritis, osteoporosis, Paget disease, corticosteroid use in the last twelve months, acute femoral neck fracture, previous surgery on the involved side, active infection, malignant tumor within the last five years, and other factors that might affect weight-bearing or rehabilitation.” [1]

• “Patients for whom the appropriate size and offset of implant were not available were also excluded.” [1]

• Score: [4] The eligibility criteria seem to be relatively inclusive, with a focus on patients with noninflammatory arthritis who are suitable for primary total hip arthroplasty. However, there are some specific exclusion criteria that may limit the generalizability of the trial.

2. Recruitment: How are participants recruited into the trial?

• “Patients were recruited from April 1997 to June 2002.” [1]

• Score: [3] The full text does not provide detailed information on how participants were recruited into the trial. It only mentions the recruitment period, which suggests that patients were likely recruited from the orthopaedic center during that time.

3. Setting: Where is the trial being done?

• “All patients underwent a primary total hip arthroplasty at a single regional orthopaedic center.” [1]

• Score: [4] The trial was conducted at a single regional orthopaedic center, which suggests that the setting of the trial closely matches the setting where the results are likely to be applied.

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “The operations were performed by the senior author (D.E.B.) in one center using the posterior approach with the patient in the lateral decubitus position.” [1]

• Score: [4] The intervention was delivered by a single experienced surgeon at a single center, which suggests that the expertise and resources needed to deliver the intervention are similar to those available in usual care.

5. Flexibility (delivery): How should the intervention be delivered?

• “All patients underwent a primary total hip arthroplasty with an SR71 or Stability femoral stem, a 28-mm cobalt-chromium Articul/eze femoral head, and an all-polyethylene cemented Elite Plus Long Posterior Wall acetabular cup.” [1]

• Score: [4] The intervention was delivered using specific components, but there is no mention of strict protocols or guidelines for how the intervention should be delivered. This suggests that there is some flexibility in the delivery of the intervention.

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• “The assessors and patients were not blinded to the implant received.” [1]

• Score: [4] The full text does not provide detailed information on measures to ensure participant adherence to the intervention. However, the lack of blinding suggests that participants were expected to adhere to the assigned intervention.

7. Follow-up: How closely are participants followed up?

• “Patients were followed for up to ten years with repeated assessments of bone mineral density, radiographs, Harris hip scores (HHS), and visual analog scale (VAS) pain scores.” [1]

• Score: [4] Participants were followed up for a relatively long period of up to ten years, with multiple assessments conducted during that time. This suggests that participants were closely followed up.

8. Primary outcome: How relevant is it to participants?

• “The primary outcome measure was bone mineral density as measured by dual x-ray absorptiometry (DXA) scan.” [1]

• Score: [5] The primary outcome measure of bone mineral density is directly relevant to participants and is a clinically important outcome in total hip arthroplasty.

9. Primary analysis: To what extent are all data included?

• “Subjects enrolled in the trial who received treatment and had at least one post-treatment assessment were included in the statistical analysis.” [1]

• Score: [4] The full text does not provide detailed information on the inclusion of all data in the primary analysis. However, the statement suggests that data from all participants who received treatment and had at least one post-treatment assessment were included, which is consistent with a pragmatic approach.

Long term effects of an integrated care intervention on hospital utilization in patients with severe COPD: a single centre controlled study (2015), PragQoL

1. Eligibility criteria: Who is selected to participate in the trial?

Quotes:

• “All participants were recruited from the DTM or the Observation Unit (OU) at the TUH. The inclusion criteria were: [1] admission due to AECOPD, [2] COPD (GOLD stage III or IV, 2007), [3] living in the Trondheim municipality, [4] an ability to communicate in Norwegian, and [5] an ability to sign the informed consent form. The exclusion criterion was: [1] any serious diseases that might cause a very short lifespan (expected survival time less than six months).”

• “All eligible patients, who gave their consent to participate in the study, were consecutively enrolled before discharge from the TUH.”

Reasoning: The eligibility criteria for this trial are clearly defined and include specific criteria related to the condition of interest (COPD), the stage of the disease (GOLD stage III or IV), and the ability to communicate and provide informed consent. The exclusion criterion also ensures that patients with a very short life expectancy are not included in the trial. Overall, the eligibility criteria appear to be pragmatic and inclusive. Score: [4]

2. Recruitment: How are participants recruited into the trial?

Quotes:

• “The study nurses contacted all patients hospitalized at the DTM or the OU because of AECOPD and invited them to participate in the study provided their home address was in the municipality of Trondheim.”

• “All participants included in the COPD-Home study were free to use all available medical services, including their GPs.”

Reasoning: Participants in the trial were recruited from patients hospitalized for AECOPD at specific healthcare facilities. The recruitment process appears to be pragmatic, as it involves contacting eligible patients during their hospital stay and inviting them to participate. Additionally, participants were free to use all available medical services, including their general practitioners. This approach ensures that the trial population is representative of patients in usual care settings. Score: [4]

3. Setting: Where is the trial being done?

Quotes:

• “All participants were recruited from the DTM or the Observation Unit (OU) at the TUH.”

• “The Trondheim municipality and the Department of Thoracic Medicine (DTM) at the Trondheim University Hospital (TUH) have created a collaborative project to develop and implement a model that integrates hospital- and municipal-based services for COPD patients based on local resources.”

Reasoning: The trial was conducted at the Department of Thoracic Medicine (DTM) at the Trondheim University Hospital (TUH) in collaboration with the Trondheim municipality. The trial setting aligns with the usual care setting for COPD patients in the Trondheim area, making it a pragmatic choice. Score: [4]

4. Organisation: What expertise and resources are needed to deliver the intervention?

Quotes:

• “The COPD-Home intervention was ambitious, and some measures were not successful. The GPs did not find time to meet with the specialist nurses in the patients homes after discharge, and a substantial number of persons were involved in the home-care services for each patient because of the continuous change of work hours among the home care nurses.”

• “At the department of thoracic medicine (DTM) at TUH the extra costs attached to the COPD-Home intervention amounted to roughly one 100% nurse position (workdays from 8 am to 3 pm) during four years including training of the home care nurses.”

Reasoning: The delivery of the COPD-Home intervention required collaboration between specialist nurses, general practitioners, and home-care services. The intervention aimed to integrate hospital- and municipal-based services for COPD patients. However, there were challenges in coordinating the involvement of different healthcare professionals and managing the work hours of home care nurses. The extra costs associated with the intervention were also mentioned. These factors suggest that the intervention required additional expertise and resources beyond usual care, reducing the pragmatic nature of the trial. Score: [3]

5. Flexibility (delivery): How should the intervention be delivered?

Quotes:

• “The core elements of the COPD-Home model were: [1] a call centre staffed by three specialist nurses for support and communication with patients and home-care nurses, and coordination between the various levels of care. The patients were routinely (at least once a month) contacted by the specialist nurses, and they were supported by telephone calls during COPD exacerbations; [2] an education session for home-care nurses: a three-hour theoretical session covering several aspects of COPD and two days of practice at the DTM; [3] an interactive 15-minute e-learning program for the patients concerning the management of COPD; [4] an individualized self-management plan introduced to the patient at discharge by the treating doctor and a specialist nurse; and [5] joint visits at the patients home by the specialist nurse (together with the home-care nurse for participants receiving home services) at approximately three days, 14 days, six months, 12 months and 24 months post-discharge.”

Reasoning: The delivery of the COPD-Home intervention involved multiple components, including a call center, education sessions, e-learning programs, individualized self-management plans, and joint visits by specialist nurses and home-care nurses. The intervention aimed to provide flexibility in how the intervention was implemented, with the details of delivery left to the providers. This approach aligns with usual care practices and suggests a pragmatic design. Score: [5]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

Quotes:

• “All participants included in the COPD-Home study were free to use all available medical services, including their GPs.”

• “The major components of these visits were repetition of the core elements of the education program, making necessary changes in the patient’s treatment plans and the reinforcing of specific health behaviors.”

Reasoning: Participants in the trial were free to use all available medical services, including their general practitioners. The joint visits by specialist nurses and home-care nurses aimed to reinforce specific health behaviors and make necessary changes in the patient’s treatment plans. These measures suggest a flexible approach to adherence, allowing participants to engage with the intervention in a way that aligns with usual care practices. Score: [5]

7. Follow-up: How closely are participants followed up?

Quotes:

• “Data from patients who completed a minimum of two years of follow-up were included in the analysis.”

• “During the two-year follow-up period, there was a trend towards an increased number of deaths in the IC group compared to the UC group, 35 (38.5%) and 21 (25.9%), respectively.”

Reasoning: Participants were followed up for a minimum of two years, and data from those who completed the follow-up period were included in the analysis. The follow-up period aligns with the duration of the intervention and allows for an assessment of long-term outcomes. However, there was a trend towards an increased number of deaths in the intervention group, which may suggest challenges in follow-up and potential differences in outcomes between the groups. Score: [4]

8. Primary outcome: How relevant is it to participants?

Quotes:

• “The primary outcomes were: [1] Number of hospital admissions caused by AECOPD (HA) and [2] number of in-hospital days (HD) due to AECOPD.”

• “HA and HD due to AECOPD include admittances where pneumonia was diagnosed during the hospital stay.”

Reasoning: The primary outcomes of the trial, number of hospital admissions and in-hospital days due to AECOPD, are directly relevant to the participants and reflect the impact of the intervention on their healthcare utilization. These outcomes align with the key question of whether the intervention reduces hospital utilization among patients with COPD. Score: [5]

9. Primary analysis: To what extent are all data included?

Quotes:

• “Data on the causes of death were analysed, and the REC concluded that the increased number of deaths in the IC group was not related to the COPD-Home intervention, but could be explained by pre-study poorer health status and higher age.”

• “The study was reopened after a break of eight months, and the project manager decided not to continue the intervention program, but continued the registration of observations in both groups until all the participants had been followed- up for three years.”

Reasoning: The primary analysis included all available data, including data on causes of death. The decision to continue the registration of observations in both groups until all participants had been followed up for three years suggests a comprehensive analysis of the data. The analysis also considered potential confounding factors, such as pre-study health status and age, when interpreting the results. Score: [5]

Range of motion of standard and high-flexion posterior cruciate-retaining total knee prostheses a prospective randomized study (2009), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

• “Fifty-four patients (mean age, 69.7 years) received a standard posterior cruciate-retaining total knee prosthesis in one knee and a high-flexion posterior cruciate-retaining total knee prosthesis in the contralateral knee.”

The full text does not provide information on the eligibility criteria used to select participants for the trial. Therefore, it is not possible to assess the PRECIS-2 score for this domain.

Score: [NA]

2. Recruitment: How are participants recruited into the trial?

• “From October to December 2004, the senior author (Y.-H.K.) performed fifty-nine consecutive primary bilateral total knee arthroplasties in fifty-nine patients.”

The full text indicates that participants were recruited consecutively during a specific time period. However, it does not provide information on any additional recruitment strategies used.

Score: [3]

3. Setting: Where is the trial being done?

• “The study was registered in the ClinicalTrials.gov Protocol Registration System (trial number, NCT00837447).”

The full text does not provide information on the setting where the trial was conducted or whether it matches the setting where the results are likely to be applied.

Score: [NA]

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “The knee was placed in a continuous-passive-motion machine after the splint was removed. All patients began walking with crutches or a walker and started active and passive range-of-motion exercises on the second day after the operation.”

The full text does not provide detailed information on the expertise and resources needed to deliver the intervention. However, it does mention the use of a continuous-passive-motion machine and the initiation of range-of-motion exercises, indicating that additional resources may have been required.

Score: [3]

5. Flexibility (delivery): How should the intervention be delivered?

• “All patients began walking with crutches or a walker and started active and passive range-of-motion exercises on the second day after the operation.”

The full text does not provide information on the flexibility of the intervention delivery. However, the mention of patients using crutches or a walker and engaging in range-of-motion exercises suggests that there may have been some flexibility in how the intervention was delivered.

Score: [3]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• “The patients used the crutches or walker, with full weight-bearing, for six weeks and used a cane when necessary thereafter.”

The full text does not provide information on specific measures to ensure participant adherence to the intervention. However, the mention of using crutches or a walker for six weeks suggests that there may have been some measures in place to promote adherence.

Score: [3]

7. Follow-up: How closely are participants followed up?

• “Clinical and radiographic evaluations were done at three months after the operation, at one year, and then yearly thereafter.”

The full text indicates that participants were followed up at specific time points after the operation. However, it does not provide information on the frequency or intensity of follow-up.

Score: [3]

8. Primary outcome: How relevant is it to participants?

• “The main goals of total knee arthroplasty are pain relief and improvement in function and the range of motion.”

The full text indicates that the primary outcome of the trial is the range of motion of the knees. This outcome is relevant to participants as it directly relates to the main goals of total knee arthroplasty.

Score: [5]

9. Primary analysis: To what extent are all data included?

• “The changes in the Knee Society, the Hospital for Special Surgery, and the WOMAC scores were evaluated with use of the Student paired t test and the Pearson nonparametric chi-square test.”

The full text does not provide information on whether all data were included in the primary analysis.

Score: [NA]

Fetoscopic laser coagulation of the vascular equator versus selective coagulation for twin-to-twin transfusion syndrome: an open-label randomised controlled trial (2014), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

Quotes:

• “Women with monochorionic, diamniotic twin pregnancies up to 26 weeks’ gestation complicated by twin-to-twin transfusion syndrome (Quintero stage16 2, 3, or 4) and women with Quintero stage 1 with clinical symptoms from polyhydramnios were eligible for participation in the trial.”

• “Exclusion criteria consisted of triplet pregnancies and mothers who were unable to provide informed consent.”

Reasoning: The eligibility criteria for this trial are clearly defined. Women with specific types and stages of twin-to-twin transfusion syndrome were included, while triplet pregnancies and mothers unable to provide informed consent were excluded. The criteria are relevant to the condition being studied and are likely to be similar to those used in usual care. [Score: 5]

2. Recruitment: How are participants recruited into the trial?

Quotes:

• “Women were informed about the study by their obstetrician after they were planned for fetoscopic laser therapy.”

• “After written informed consent was obtained, women were randomly assigned (1:1) to the Solomon technique or standard laser coagulation by online randomisation.”

Reasoning: Participants were recruited through their obstetricians after being planned for fetoscopic laser therapy. The recruitment process seems to be similar to how patients would be recruited in usual care. [Score: 5]

3. Setting: Where is the trial being done?

Quotes:

• “We did an open-label, international, multicentre, randomised controlled trial at five European tertiary referral centres.”

• “The trial was funded by the Netherlands Organization for the Health Research and Development.”

Reasoning: The trial was conducted at five European tertiary referral centers, indicating that the trial was done in a setting that is likely to be similar to where the results will be applied. The international and multicenter nature of the trial also increases its applicability. [Score: 5]

4. Organisation: What expertise and resources are needed to deliver the intervention?

Quotes:

• “All fetoscopic laser procedures were undertaken by experienced operators (12 surgeons), each of whom had done at least 60 previous laser procedures and were competent to undertake the Solomon technique.”

• “All five participating centres used the same fetoscopic laser coagulation technique as described previously.”

Reasoning: The intervention was delivered by experienced operators who had performed a sufficient number of previous laser procedures. The expertise and resources needed to deliver the intervention seem to be similar to what would be available in usual care. [Score: 5]

5. Flexibility (delivery): How should the intervention be delivered?

Quotes:

• “The following steps were identical for both the experimental and the standard procedure.”

• “Both groups received identical postoperative monitoring and peripartum management.”

Reasoning: The delivery of the intervention was standardized and identical for both the experimental and standard procedures. The postoperative monitoring and peripartum management were also identical for both groups. This indicates a pragmatic approach to the delivery of the intervention, as it closely resembles usual care. [Score: 5]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

Quotes:

• “Follow-up visits, including an ultrasound assessment, were scheduled a minimum of once every 2 weeks.”

• “The timing and mode of delivery were at the discretion of each woman’s obstetric caregiver.”

Reasoning: The follow-up visits were scheduled at a frequency that is similar to what would be done in usual care. The timing and mode of delivery were also left to the discretion of each woman’s obstetric caregiver, indicating a pragmatic approach to adherence. [Score: 5]

7. Follow-up: How closely are participants followed up?

Quotes:

• “Follow-up visits, including an ultrasound assessment, were scheduled a minimum of once every 2 weeks.”

• “Neonatal follow-up was done by the treating neonatologist. Follow-up was until term age or at discharge (whichever came first); neonatal outcome in this study only included short-term outcome.”

Reasoning: Participants were followed up with regular visits and ultrasound assessments, which is similar to what would be done in usual care. Neonatal follow-up was done by the treating neonatologist until term age or discharge. The follow-up seems to be consistent with usual care. [Score: 5]

8. Primary outcome: How relevant is it to participants?

Quotes:

• “The primary outcome was a composite endpoint consisting of the following four items: incidence of twin anaemia polycythaemia sequence, recurrence of twin-to-twin transfusion syndrome, perinatal mortality, or severe neonatal morbidity.”

• “The primary outcome was scored in case one of the endpoints was reached.”

Reasoning: The primary outcome is a composite endpoint that includes outcomes that are relevant to the participants, such as twin anaemia polycythaemia sequence, recurrence of twin-to-twin transfusion syndrome, perinatal mortality, and severe neonatal morbidity. The primary outcome is scored if any of these endpoints are reached. [Score: 5]

9. Primary analysis: To what extent are all data included?

Quotes:

• “Analyses were done by intention to treat.”

• “All statistical data were analysed using SPSS version 20.0 (IBM, Chicago, IL, USA).”

Reasoning: The primary analysis was done by intention to treat, which is a pragmatic approach that includes all data. The statistical analysis was done using a commonly used software. [Score: 5]

The safety and efficacy of a paclitaxel-eluting wrap for preventing peripheral bypass graft stenosis: a 2-year controlled randomized prospective clinical study (2008), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

Quotes:

• “Hundred and nine subjects were randomized 2:1 to treatment or control.”

• “All subjects were implanted with a 6 mm expanded PTFE vascular graft; in addition, treated subjects had a 2.5 cm × 4 cm paclitaxel-eluting wrap (1.6 μg/mm2) placed around the distal graft anastomosis.”

Reasoning: The full text provides information on the eligibility criteria for the trial. It states that 109 subjects were randomized to treatment or control groups. All subjects received a 6 mm expanded PTFE vascular graft, and treated subjects also received a paclitaxel-eluting wrap placed around the distal graft anastomosis. The eligibility criteria are clearly described, indicating that the trial included patients scheduled to undergo femoropopliteal peripheral bypass surgery with a PTFE graft.

Score: [5]

2. Recruitment: How are participants recruited into the trial?

Quotes:

• “Study subjects provided written informed consent prior to their enrollment into this study.”

• “Each investigative site received a set of numbered computer-generated sealed randomization envelopes, which were opened in sequence during the surgery, following hemostasis at the anastomosis and prior to wound closure.”

Reasoning: The full text mentions that study subjects provided written informed consent prior to enrollment, indicating that they were recruited through the process of informed consent. It also mentions the use of randomization envelopes opened during surgery, suggesting that participants were recruited at the time of their scheduled surgery.

Score: [4]

3. Setting: Where is the trial being done?

Quotes:

• “This study was conducted at nine investigational sites in Hungary, The Netherlands and The Netherlands Antilles.”

Reasoning: The full text specifies that the study was conducted at nine investigational sites in Hungary, The Netherlands, and The Netherlands Antilles. This information provides clarity on the setting of the trial.

Score: [4]

4. Organisation: What expertise and resources are needed to deliver the intervention?

Quotes:

• “Using standard surgical technique, the femoral artery and the popliteal artery were exposed.”

• “The arteriotomy and proximal anastomosis were performed using standard techniques.”

• “If the subject was randomized to the treatment group, the backing layer was first removed from the wrap, and then the wrap was placed around the distal end of the anastomosis.”

Reasoning: The full text mentions the use of standard surgical techniques for the implantation of the graft and the placement of the wrap. This suggests that the intervention can be delivered using existing expertise and resources without the need for additional training or specialized procedures.

Score: [5]

5. Flexibility (delivery): How should the intervention be delivered?

Quotes:

• “If the subject was randomized to the treatment group, the backing layer was first removed from the wrap, and then the wrap was placed around the distal end of the anastomosis.”

• “The wrap was positioned to cover the popliteal artery including the toe and heel of the graft.”

Reasoning: The full text describes the delivery of the intervention, stating that the wrap was placed around the distal end of the anastomosis and positioned to cover the popliteal artery. The description suggests that there is flexibility in how the intervention is delivered, as it does not provide specific instructions for the exact placement or technique of applying the wrap.

Score: [4]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

Quotes:

• “No formal statistical tests were planned or performed on any of the demographic or safety variables; this study was not designed to detect quantitative differences in these measures.”

• “No adverse events were judged to be related to the use of the investigational device.”

Reasoning: The full text does not provide specific information on measures to ensure participant adherence to the intervention. However, it does mention that no formal statistical tests were performed on demographic or safety variables, indicating that adherence was not a primary focus of the study. Additionally, it states that no adverse events were judged to be related to the use of the investigational device, suggesting that participants adhered to the intervention without significant issues.

Score: [3]

7. Follow-up: How closely are participants followed up?

Quotes:

• “These assessments were repeated during subject follow-up visits at Week 6 and at Months 3, 6, 12, and 18.”

• “At Month 24, safety and performance evaluations included the above-listed assessments and clinical laboratory tests.”

Reasoning: The full text mentions follow-up visits at specific time points, including Week 6 and Months 3, 6, 12, 18, and 24. It also states that safety and performance evaluations were conducted at Month 24, which included assessments and clinical laboratory tests. This indicates that participants were followed up closely over a 24-month period.

Score: [4]

8. Primary outcome: How relevant is it to participants?

Quotes:

• “The primary safety measure was reported AEs.”

• “The internal diameter of the distal graft anastomoses, as measured by ultrasound, served as the primary efficacy measure.”

Reasoning: The full text mentions that the primary safety measure was reported adverse events (AEs) and that the primary efficacy measure was the internal diameter of the distal graft anastomoses measured by ultrasound. These outcomes are directly relevant to the participants and reflect the safety and effectiveness of the intervention.

Score: [5]

9. Primary analysis: To what extent are all data included?

Quotes:

• “Descriptive statistics were performed on all demographic, safety, and performance variables.”

• “No formal statistical tests were planned or performed on any of the demographic or safety variables; this study was not designed to detect quantitative differences in these measures.”

Reasoning: The full text states that descriptive statistics were performed on all demographic, safety, and performance variables. It also mentions that no formal statistical tests were planned or performed on these variables. This suggests that all available data were included in the analysis, indicating a pragmatic approach.

Score: [5]

Laparoscopic versus open approach for aortobifemoral bypass for severe aorto-iliac occlusive disease–a multicentre randomised controlled trial (2012), PragSurgery

1. Eligibility criteria: Who is selected to participate in the trial?

• “Between January 2007 and November 2009, 28 patients with severe aorto-iliac occlusive disease (TASC II C or D) were randomised between laparoscopic and open approach at one community hospital and one university hospital.”

• “Exclusion criteria were as follows: combined aneurysmal-occlusive disease, impossible infrarenal clamping based on preoperative computed tomography (CT) scan, Fontaine IVb, age >80 years, creatinine >3.0 mg dl−1, severe pulmonary insufficiency (partial pressure of oxygen in the blood (PaO2) < 60 mmHg or partial pressure of carbon dioxide in the blood (PaCO2) > 50 mmHg on room air), severe cardiac dysfunction (myocardial infarction in past 6 months, ejection fraction <35% and severe aortic valve stenosis), previous aortic or major abdominal surgery, previous abdominal radiation therapy or severe pancreatitis, failed epidural anaesthesia or any contra-indication for epidural or general anaesthesia and patients not willing to sign informed consent.”

• “Patients were excluded if they did not meet the inclusion criteria or presented in an emergency setting.”

The full text provides information on the inclusion and exclusion criteria for the trial. Patients with severe aorto-iliac occlusive disease (TASC II C or D) were included, while patients with certain comorbidities or contraindications were excluded. The eligibility criteria appear to be pragmatic, as they include patients who would typically receive the intervention in usual care and exclude those who may not be suitable candidates. Score: [4]

2. Recruitment: How are participants recruited into the trial?

• “Between January 2007 and November 2009, all consecutive patients necessitating an aortobifemoral bypass for severe aorto-iliac occlusive disease TASC II C or D at Hôpital St. Joseph, Charleroi, Belgium or at University Hospitals Leuven, Leuven, Belgium, were considered for random assignment to either laparoscopic or conventional open aortobifemoral bypass surgery.”

• “Assignment to laparotomy or laparoscopy was obtained using ‘www.randomization.com’. This free generator randomises each subject to a single treatment by using the method of randomly permuted blocks.”

The full text states that consecutive patients who required aortobifemoral bypass surgery for severe aorto-iliac occlusive disease were considered for random assignment to either laparoscopic or conventional open surgery. The recruitment process appears to be pragmatic, as it involves patients who would typically receive the intervention in usual care. Score: [5]

3. Setting: Where is the trial being done?

• “Between January 2007 and November 2009, all consecutive patients necessitating an aortobifemoral bypass for severe aorto-iliac occlusive disease TASC II C or D at Hôpital St. Joseph, Charleroi, Belgium or at University Hospitals Leuven, Leuven, Belgium, were considered for random assignment to either laparoscopic or conventional open aortobifemoral bypass surgery.”

The full text mentions that the trial was conducted at Hôpital St. Joseph in Charleroi, Belgium and University Hospitals Leuven in Leuven, Belgium. The trial was conducted in real-world clinical settings, which suggests a pragmatic approach to the choice of setting. Score: [5]

4. Organisation: What expertise and resources are needed to deliver the intervention?

• “A standardised operative technique was used.”

• “For all patients there was a standardised protocol for anaesthesia and patient-controlled epidural anaesthesia (PCEA).”

• “Perioperative antibiotics, intravenous cefazolin or vancomycin for penicillin-allergic patients, were administered for 24 h.”

The full text mentions the use of a standardised operative technique, standardised protocols for anaesthesia and patient-controlled epidural anaesthesia, and the administration of perioperative antibiotics. These aspects suggest a pragmatic approach to the organisation of care, as the intervention was delivered using existing healthcare staff and resources. Score: [5]

5. Flexibility (delivery): How should the intervention be delivered?

• “A standardised operative technique was used.”

• “The type of aorto-prosthetic anastomosis (end-to-end or end-to-side) is decided based on the quality of the aorta as estimated on the preoperative CT scan.”

The full text mentions the use of a standardised operative technique and the decision on the type of aorto-prosthetic anastomosis based on the quality of the aorta. These aspects suggest a pragmatic approach to the delivery of the intervention, as the details of how the intervention is performed are not rigidly prescriptive. Score: [5]

6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

• No specific measures mentioned in the full text.

The full text does not provide information on measures to ensure participant adherence to the intervention. Without this information, it is not possible to assess the flexibility of adherence in the trial. Score: [NA]

7. Follow-up: How closely are participants followed up?

• “Patients were followed up clinically twice daily or more frequently as needed during their hospitalisation until discharge.”

• “Patients were followed up in the outpatient vascular surgery clinic at 2 weeks and 1 month postoperatively.”

The full text mentions that patients were followed up clinically during their hospitalisation until discharge and had follow-up appointments at 2 weeks and 1 month postoperatively. The frequency of follow-up appears to be pragmatic, as it aligns with usual care practices. Score: [4]

8. Primary outcome: How relevant is it to participants?

• “To investigate differences between open and laparoscopic aortobifemoral bypass surgery for aorto-iliac occlusive disease on postoperative morbidity and mortality.”

The full text states that the primary outcome of the trial was to investigate differences in postoperative morbidity and mortality between open and laparoscopic aortobifemoral bypass surgery. This outcome is relevant to participants as it directly relates to their health outcomes. Score: [5]

9. Primary analysis: To what extent are all data included?

• “Data were analysed on an intention-to-treat basis.”

The full text mentions that the data were analysed on an intention-to-treat basis, which suggests that all data were included in the analysis. This approach aligns with a pragmatic design. Score: [5]

System prompt

You are an expert in clinical trial design and are tasked to assess the PRECIS-2 scores of a trial based on the full text of its publication. You have the following briefing:
---
# The nine domains of PRECIS-2

## 1. Eligibility criteria: Who is selected to participate in the trial?

A highly pragmatic approach to eligibility criteria would be to include in the trial anyone with the condition of interest who is likely to be a candidate for the intervention if it was being provided in usual care for this condition. So, if in usual care the intervention would likely be used for children, elderly people, people with a range of comorbidities, and people with a probable diagnosis rather than, say, a laboratory confirmed diagnosis, then all of these groups should be in the trial too. If people aged over 80 or with multiple morbidities would not receive the intervention in usual care because we already know that this intervention does more harm than good in elderly or multi-morbid patients, then a pragmatic design choice would be to exclude them from the trial. A trial with eligibility design choices like this would score at or close to 5 on the PRECIS-2 scale.

As the level of similarity between the people in the trial and those in usual care decreases, then so would the PRECIS-2 score. Many things could reduce the score, but some examples include

* Excluding people not known or shown to be highly adherent to the intervention.
* Excluding people using tests or measures that are not used or available in usual care.
* Excluding people not expected to be highly responsive to the intervention.
* Excluding people who would be likely to receive the intervention in usual care but inclusion in the trial is considered too difficult because of challenges unrelated to the delivery of the intervention. Children, people over the age of 65, and pregnant women are good examples of people commonly excluded because of anticipated difficulties of gaining ethical approval rather than whether they would be likely to receive an intervention in usual care.
* Excluding people whose adherence and follow-up may pose difficulties, whether because of social disadvantage, personal circumstances, or illness (such as homelessness, substance misuse, mental health problems, or poor motivation).
* Excluding people dependent on help for activities of daily living, unless they would usually not be eligible for intervention.

Finally, exclusion criteria for known safety issues would not generally reduce the PRECIS-2 score since such individuals would not be expected to get the intervention in usual care.

## 2. Recruitment: How are participants recruited into the trial?

This domain has been included to encourage trialists to consider how and where they will recruit people to their trial.

The most extreme pragmatic approach to recruitment would be to do this in usual care so that only the people who attend a clinic with the condition of interest are recruited after they present on their own behalf without any overt recruitment effort. A highly pragmatic approach to trial design would also recruit from more than one clinic as an explicit way of increasing applicability of the trial results. Recruiting through usual appointments at a diverse range of clinics is likely to score at or close to 5 on the PRECIS-2 scale.

If the above is not feasible, at least as the sole recruitment route, trialists will need to add other recruitment strategies. If extra resources will be required to recruit people to the trial, trialists need to consider if this affects applicability, especially if participants are recruited from settings other than the one in which the results are likely to be applied. For example, recruiting in secondary care for an intervention that will mainly be used in primary care will ensure that the recruited patients are different in severity and treatment than the patients with this same condition in primary care; and so this will make the trial more explanatory and reduce the PRECIS 2 score. Additional recruitment approaches that could potentially reduce the PRECIS-2 score include

* Searching medical record systems for eligible participants and then mailing invitation letters. Depending on local resources, such as electronic medical records and comprehensive local disease registers, this approach may be easily achievable in usual care and so would not reduce the PRECIS-2 score.
* Media advertising campaigns such as television and radio advertisements, newspaper advertisements, flyers, websites, press releases, letters to the editor, interviews with the chief investigator on television or radio. The issue to consider here is whether these campaigns are being used simply to speed up recruitment, or because they are the only way to reach a dispersed population. In the latter case, if such campaigns would not be available in usual care then this trial recruitment route is more explanatory.
* Incentives such as cash payment, vouchers, travel costs, or childcare. Some of these incentives may be better considered as part of the intervention (for example, a lifestyle change intervention aimed at new mothers may only be effective because childcare was also provided and women could therefore attend sessions).

## 3. Setting: Where is the trial being done?

This domain has been included to encourage trialists to explicitly consider the match between the setting of their trial and the setting where their results are likely to be applied. While there are no accepted guidelines for how to assess the impact of setting on applicability, there are several characteristics of the setting that will almost always need to be considered, including geography, healthcare system, country, and the socioeconomic and ethnic mix of the population. All of these might affect the applicability of results from a particular trial.

The most extremely pragmatic approach to setting would be to do the trial in an identical setting to which you intend the results to be applied. Such a trial is likely to score at or close to 5 on PRECIS-2.

Even settings that seem rather restrictive could still be a highly pragmatic design choice if this setting is the usual care setting where patients are treated for the particular health condition. For example, if a trial was carried out in the most specialist intensive care units in the country and the intention of the trial was to support decision making in these highly specialised units, then the design choice with regard to setting is still pragmatic. Conversely, if the trial results were intended to be useful for all intensive care units, but the trial included only highly specialised units, then the trial is now taking a more explanatory approach to the choice of setting that goes against the intention of the trial.

Approaches to setting that are likely to reduce the PRECIS-2 score include

* Selecting participating centres from among only specialist or academic centres when the trial is intended to be applicable to all types of practice or clinic treating the condition of interest.
* Running the trial in a single centre. It is rare that a trial is done with the intention that its results should apply to only that single centre. Including several centres makes it easier to claim that the trial setting matches that to which the results will be applied.

## 4. Organisation: What expertise and resources are needed to deliver the intervention?

This domain has been included to encourage trialists to consider the match between how care is organised and delivered in the trial and how the intervention would be made available to patients in usual care. As there have been acknowledged difficulties in implementing results post-trial (even in settings where the trial was conducted), we wanted trialists to consider organising the delivery of their trial intervention with an awareness of how easy it would be to implement the intervention post-trial .

A highly pragmatic design would aim to slot the intervention into the usual organisation of care for the condition of interest, making use of no more than the existing healthcare staff and resources in that setting. A design like this is likely to score at or close to 5 on PRECIS-2.

Sometimes a change to how care is organised is itself the intervention being evaluated, and the difference between the trial and usual care is clear. In this case the organisation domain would be explanatory compared with current usual care, but trialists can note that the difference is an explicit feature of the intervention.

The PRECIS-2 organisation score will be reduced to the degree trialists make changes in how care is delivered compared with the usual recipients and care delivery. For example, the trial team may, in the intervention arm, provide additional training to staff or provide additional staff so that the intervention can be delivered. Patients who would normally be seen in primary care may be seen by specialists in secondary care instead, or they may receive additional expensive diagnostic procedures unavailable in usual care. The success or otherwise of the trial then depends on resources unavailable in usual care, reducing applicability. If these additional resources are essential to the effectiveness of the intervention they should be considered to be part of the intervention and be specified in the part of the protocol describing the intervention. In that case these resources should not be counted when this element of the trial is judged.

Organisational approaches that are likely to reduce the PRECIS-2 score include the following (but only when they are not incorporated and described as necessary elements of the intervention):

* Increasing the number of healthcare or other professionals available to deliver the intervention over and above the levels available in usual care.
* Providing significant levels of additional training to increase the expertise of healthcare professionals.
* Requiring healthcare professionals to have some minimum level of experience, defined by length of time, in working with the intervention that is greater than would be the case in usual care.
* Requiring healthcare professionals to have a specialty certification that would not be considered essential to deliver the intervention in usual care.
* Increasing the resources (such as facilities, diagnostic equipment, consumables) available to deliver the intervention, measure outcomes, or do follow-up over and above what would be available in usual care.

## 5. Flexibility (delivery): How should the intervention be delivered?

In PRECIS-2, trialists wishing to conduct more pragmatic trials are encouraged to think about how the intervention will be implemented post-trial in the setting where the results are expected to be applied. The most pragmatic design approach to delivery flexibility would leave the details of how to implement the intervention up to providers, in other words, what happens in usual care. Thus, the methodology of how to deliver an intervention is not rigidly prescriptive in the protocol. For example, the details of how to perform a surgical procedure could be left entirely to the surgeon, or how to deliver an educational programme is left to the discretion of the educator. Additionally, a pragmatic approach would not dictate which other interventions were permitted, or how to deliver them, which again is the situation in usual practice. This sort of delivery flexibility would score at or close to 5 on PRECIS-2. Flexibility (delivery) applies to all intervention arms and the comparator (control) arm if usual care is not the comparator, and if necessary each of these arms will need to be scored separately. If usual care is the intended comparator but any elements relating to delivery are changed it ceases to be usual care and needs to be scored.

As delivery flexibility is reduced, the trial moves towards a more explanatory approach in this domain. Approaches to delivery flexibility that are likely to reduce the PRECIS-2 score include

* A highly specified, protocol driven intervention. For example, specific direction is given for how to the administer the intervention (such as dose, dosing schedule, surgical procedures, educational material, and delivery)
* Having measures in place to monitor the compliance of those delivering the intervention (such as doctors) with the protocol and measures (up to and including exclusion) to address poor compliance.
* The timing of intervention delivery is tightly defined and designed to maximise the intervention effect.
* Providers undertake additional interventions that would not occur in usual care
* Restrictions are placed on the number and types of co-interventions, particularly if excluded co-interventions would dilute any intervention effect
* There is specific direction for applying permitted co-interventions
* There are specific directions for managing complications or side effects of the intervention.

## 6. Flexibility (adherence): What measures are in place to ensure participants adhere to the intervention?

It is important that trialists focused on applicability consider how probable it is that there will be uptake of the intervention being tested and how recipients will engage with it in the setting in which it will be later applied. A highly pragmatic design approach would allow for full flexibility in how end user recipients engage with the intervention. In usual care, health professionals encourage patients to take medication or follow therapy as best they can, and such encouragement would not count against a pragmatic design; if it also happens in usual care, allowing it in the trial is a pragmatic design decision. A trial with no special measures to enforce engagement or compliance would score at or close to 5 on PRECIS-2. On the other hand, a trial protocol that lays out methods to monitor and ensure patient compliance would score at or close to 1 on PRECIS-2. Flexibility (adherence) applies to all intervention arms and the comparator (control) arm if usual care if not the comparator, and if necessary each of these arms will need to be scored separately. If usual care is the intended comparator but any elements relating to adherence are changed it ceases to be usual care and needs to be scored.

It is possible that improving recipients’ ability to take medication or follow therapy is the intervention under evaluation, a situation similar to that already described for the Setting and Organisation domains. In this case, the Flexibility (adherence) domain would be explanatory compared with current usual care, but trialists can note that the difference is an explicit feature of the intervention.

As flexibility for recipients is reduced, the trial moves towards a more explanatory approach on this domain. Approaches to recipient flexibility that are likely to reduce the PRECIS-2 score include

* Having a trial pre-screening stage where patients are evaluated for adherence with the intervention (such as dose, dosing schedule, attendance at therapy sessions). Patients judged to not be adherent are excluded.
* Withdrawing patients from the trial if their adherence with the intervention drops below some specified level. For example, a trial might withdraw patients if they failed to attend two consecutive sessions of cognitive behavioural therapy or failed to take more than 90% of their medication.
* Having measures in place to monitor patient adherence with the protocol and measures (up to and including withdrawal) to address poor adherence. Measures to increase patient adherence should this fall below some specified level could, for example, include scheduling an extra discussion with a research nurse about why following the trial instructions is important.

## 7. Follow-up: How closely are participants followed up?

This domain encourages trialists to think about follow-up with regard to the setting where the results will be applied. The most pragmatic position with regard to follow-up would be to have no more follow-up of recipients than would be the case in usual care. Indeed, the most extreme position is to have no follow-up contact at all with recipients and to obtain outcome data by other means (such as electronic medical records or other usual data to measure mortality or hospital admissions). Trials that have no more follow-up than is normal in usual care and have minimal additional data collection (or use data obtained from administrative or clinical record systems without direct contact with the participant) would be likely to score at or close to 5 on PRECIS-2.

As follow-up becomes more intense, the trial becomes more explanatory and the PRECIS-2 score will decrease. Follow-up approaches that are likely to reduce the PRECIS-2 score include

* Follow-up visits that are more frequent than would typically occur under usual care.
* Unscheduled follow-up visits are triggered by a primary outcome event.
* Unscheduled follow-up visits are triggered by an intervening event that is likely to lead to the primary outcome event.
* Patients are contacted if they fail to keep trial appointments. This would not reduce the PRECIS-2 score if this was also done in usual care.
* More extensive data are collected, particularly intervention related data, than would be typical outside the trial
* Visits are longer than usual care and involve additional or different staff.

Often the required trial outcomes may be obtained only through contact with the recipients. Even in the “no follow-up” approach, assessment of outcomes may be achieved with a single follow-up at the end of the study. The end of study would need to be defined so that there is sufficient time for the desired study outcomes (see Primary outcome domain below) to be observed. When the follow-up is done in this way, it is unlikely to have an impact on engagement or response to the treatment, and so such a follow-up approach would be viewed as pragmatic and rated at or close to 5.

It is often the case that explanatory trials are interested in the effect of an intervention only during a brief intervention period or shortly after. On the other hand, trials taking a pragmatic approach to design may follow recipients well beyond the intervention period. This longer period of follow-up may require more contacts than would be the case in usual care, but this need not be inconsistent with a pragmatic approach if it does not result in care management that differs from the usual conditions. It is, however, important to consider the burden of follow-up for participants.

## 8. Primary outcome: How relevant is it to participants?

The choice of primary outcome is a crucial trial design decision, and a pragmatic approach would be to select an outcome that is of obvious importance from the patient’s perspective. Post-trial, an outcome selected using a pragmatic approach would also be relevant to commissioners of care, the people who decide whether to implement the intervention on the basis of its results. For example, an intervention that aims to reduce falls in elderly people living independently in the community should have as its primary outcome the number of falls in the elderly living independently in the community. This outcome has meaning to patients, their relatives and friends, healthcare professionals, and policymakers. Measures of, say, bone density, muscle strength, or functional ability are distant from the key question of whether the intervention prevents elderly people falling in their own homes. Trials that choose outcomes of obvious importance to patients, and measure them in a way that is the same or similar to the way they are measured in usual care would be likely to score at or close to 5 on PRECIS-2.

As the primary outcome becomes less recognisably important to patients, or is assessed on criteria seldom used in usual care, the trial becomes more explanatory and the PRECIS-2 score will decrease. Approaches to the primary outcomes that are likely to reduce the PRECIS-2 score include

* Choosing a surrogate outcome (such as blood test) that the intervention is expected to have a direct effect on, or the use of safety climate surveys rather than medical error events or near miss events.
* Using a composite primary outcome in which some of the elements are less important to patients or participants than others (such as incidence of death, stroke, myocardial infarction, or renal dysfunction at 30 days).
* Having central adjudication of the outcome or using an assessment that needs special training or tests not normally used in usual care.
* Choosing an outcome that is important but mainly to providers, typically because it is a physiological outcome considered useful in treatment planning and monitoring.
* Measuring an outcome that is important but at a time that is earlier than would be normal in usual care. For example, stroke could be a primary outcome in trials making explanatory or pragmatic design choices. However, time horizons may vary from short term after a one-time intervention (more explanatory) to long term (more pragmatic).

## 9. Primary analysis: To what extent are all data included?

Most trials are a superiority design so the most pragmatic approach with regard to the analysis would be to make no special allowance in the analysis for non-adherence, practice variability, etc. In other words, the pragmatic approach to the analysis would typically be an intention-to-treat analysis using all available data. This sort of analysis would score at or close to 5 on PRECIS-2. (Non-inferiority and equivalence trials are usually contrary to intention-to-treat analysis.)

The intention-to-treat analysis has recently also become the norm for superiority trials taking a more explanatory design approach, especially when regulatory approval for an intervention is being sought. The most explanatory approach would be to use the “as treated analysis” as the primary analysis, in which only those patients who actually received (or did not receive) the intervention would be analysed in the intervention and control groups respectively, irrespective of their initial randomised group allocation.

Other causes of missing data may be important but may not affect how pragmatic or explanatory a trial is. Systematic exclusion of data from participants because, say, they were poorly adherent would make a trial more explanatory. Using all data but doing nothing to try and fill gaps caused by missing data would not in itself make trial more pragmatic or explanatory; missing data, especially if there is a lot of it, makes any conclusions more uncertain regardless of the design approach taken.

Approaches to the primary analysis that are likely to reduce the PRECIS-2 score include

* Excluding non-compliant recipients (per protocol analysis)
* Analyse recipients to treatment received instead of treatment randomised (as treated analysis)
* Excluding data from non-adherent providers
* Excluding data from trial sites or providers who recruit below expected volume.

For some trials taking an explanatory approach (dose finding trials are an example), it may be appropriate to have primary analysis restricted in the ways mentioned; otherwise such restricted analyses of the primary outcome could be pre-planned as secondary analyses of the primary outcome.
---

User prompt

This is the full text to be assessed:
---
%FULLTEXT%
---

For each of the 9 domains of PRECIS-2, perform the following steps:
1. Extract 1-3 relevant quotes from the full text.
2. Explain your reasoning in 1 paragraph.
3. Give a score X from 1 (very explanatory) to 5 (very pragmatic) in square brackets like this: "Score: [X]". If the full text does not contain enough information to assess a specific domain, report "Score: [NA]".

Do not provide a final summary paragraph in the end.